diff options
| author | Jasper Ras <jras@hostnet.nl> | 2025-04-11 16:02:40 +0200 |
|---|---|---|
| committer | Jasper Ras <jras@hostnet.nl> | 2025-04-11 16:02:40 +0200 |
| commit | eae6b57f17f5065a2bd3f130668043693f579cb4 (patch) | |
| tree | 7b695dd512530d81790063e2ee769eab2ba560e0 | |
| parent | f9034731fa234a4b5efa1d2d6147fe1e798b6d36 (diff) | |
vault backup: 2025-04-11 16:02:40
23 files changed, 286 insertions, 59 deletions
diff --git a/.obsidian/daily-notes.json b/.obsidian/daily-notes.json new file mode 100644 index 0000000..3efcc00 --- /dev/null +++ b/.obsidian/daily-notes.json @@ -0,0 +1,4 @@ +{ + "folder": "/daily", + "format": "DD-MMM-Y" +}
\ No newline at end of file diff --git a/.obsidian/workspace.json b/.obsidian/workspace.json index f667601..9818d14 100644 --- a/.obsidian/workspace.json +++ b/.obsidian/workspace.json @@ -8,67 +8,37 @@ "type": "tabs", "children": [ { - "id": "502f77e94f5ff48c", + "id": "8533accf5c2515ac", "type": "leaf", "state": { "type": "markdown", "state": { - "file": "Managing Ceph on Devstack & OpenStack.md", + "file": "daily/11-Apr-2025.md", "mode": "source", "backlinks": true, "source": false }, "icon": "lucide-file", - "title": "Managing Ceph on Devstack & OpenStack" + "title": "11-Apr-2025" } }, { - "id": "c10897c003c6bee7", + "id": "a06a83491e7620e5", "type": "leaf", "state": { "type": "markdown", "state": { - "file": "OVN upgrade.md", + "file": "Device mapper.md", "mode": "source", "backlinks": true, "source": false }, "icon": "lucide-file", - "title": "OVN upgrade" - } - }, - { - "id": "62d186e3d63b15e0", - "type": "leaf", - "state": { - "type": "markdown", - "state": { - "file": "I'm worried that Freenet will completely go DOWN due to OVN upgrade.md", - "mode": "source", - "backlinks": true, - "source": false - }, - "icon": "lucide-file", - "title": "I'm worried that Freenet will completely go DOWN due to OVN upgrade" - } - }, - { - "id": "96a8cc05fb9592a6", - "type": "leaf", - "state": { - "type": "markdown", - "state": { - "file": "Network outage on single hypervisor causes Freenet to shutdown EVERYTHING.md", - "mode": "source", - "backlinks": true, - "source": false - }, - "icon": "lucide-file", - "title": "Network outage on single hypervisor causes Freenet to shutdown EVERYTHING" + "title": "Device mapper" } } ], - "currentTab": 3 + "currentTab": 1 } ], "direction": "vertical" @@ -99,7 +69,7 @@ "state": { "type": "search", "state": { - "query": "OVN upgrade", + "query": "rbd", "matchingCase": false, "explainSearch": false, "collapseAll": false, @@ -219,34 +189,35 @@ "obsidian-excalidraw-plugin:New drawing": false } }, - "active": "96a8cc05fb9592a6", + "active": "a06a83491e7620e5", "lastOpenFiles": [ - "Making notes useful.md", + "daily/11-Apr-2025.md", + "Device mapper.md", + "daily/10-Apr-2025.md", + "List of tags I use in this Vault and their purpose.md", "Managing Ceph on Devstack & OpenStack.md", - "Network outage on single hypervisor causes Freenet to shutdown EVERYTHING.md", + "OpenStack Tempest.md", + "Tempest Plugin.md", + "High Gear Low Gear Testing.md", + "TDD.md", + "The volume type of a Cinder volume can be changed.md", + "How to (RIP) Grep.md", + "I assumed that Mohammed made an oopsie but instead it turned out to be one of us that forgot to clean up.md", + "Writing a test double before the real implementation gives feedback on design.md", + "London-style versus Detroit-style TDD.md", "I'm worried that Freenet will completely go DOWN due to OVN upgrade.md", - "Random notes on Freenet deployment.md", "OVN upgrade.md", + "daily", + "2025-04-10.md", "Installing Ceph on devstack.md", "Overview of Ceph.md", "Gobs Ceph Client Key Access.md", - "Installing a devstack plugin.md", - "Rewriting subselect into just an additional join.md", - "Updating the image os admin user to fix password resets.md", - "Freenet CPU steal graph.md", - "Using Libvirt to manage QEMU image bitmaps.md", - "Build custom modules as profiles to manage various NixOS hosts.md", - "Nix Inherit.md", - "Becoming a Senior.md", - "A reason for becoming senior is that when Rutger is off I take most of his responsibilities.md", - "Acronis Managed Backup provisioning flow.md", - "Ceph.md", - "NixOS - Imperative container management.md", - "NixOS - Declarative container management.md", - "NixOS - Containers.md", - "NixOS - Container networking.md", - "EF02 partition.md", - "EF00 partition.md", + "Detroit-style TDD.md", + "Blank Page Syndrom is when you start with infinite choice and cannot make a choice.md", + "London-style TDD.md", + "Discovery Testing.md", + "YAGNI.md", + "Network outage on single hypervisor causes Freenet to shutdown EVERYTHING.md", "ESP", "Excalidraw" ] diff --git a/.trash/2025-04-10.md b/.trash/2025-04-10.md new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/.trash/2025-04-10.md diff --git a/.trash/Blank Page Syndrom is when you start with infinite choice and cannot make a choice.md b/.trash/Blank Page Syndrom is when you start with infinite choice and cannot make a choice.md new file mode 100644 index 0000000..417bc2c --- /dev/null +++ b/.trash/Blank Page Syndrom is when you start with infinite choice and cannot make a choice.md @@ -0,0 +1 @@ +A way to counter this is by showing an example instead of a blank page.
\ No newline at end of file diff --git a/.trash/I assumed that Mohammed made an oopsie but instead it turned out to be one of us that forgot to clean up.md b/.trash/I assumed that Mohammed made an oopsie but instead it turned out to be one of us that forgot to clean up.md new file mode 100644 index 0000000..9e1cee6 --- /dev/null +++ b/.trash/I assumed that Mohammed made an oopsie but instead it turned out to be one of us that forgot to clean up.md @@ -0,0 +1,6 @@ +--- +tags: + - self + - reflection +--- + diff --git a/.trash/Untitled 2.md b/.trash/Untitled 2.md new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/.trash/Untitled 2.md diff --git a/Detroit-style TDD.md b/Detroit-style TDD.md new file mode 100644 index 0000000..5726068 --- /dev/null +++ b/Detroit-style TDD.md @@ -0,0 +1,18 @@ +--- +tags: + - software-design + - software-engineering + - tdd +--- +Classic TDD: + +1. Write a failing test for public API (red) +2. Change implementation to make it pass (green) +3. Refactor +4. Go to step 1 + +Benefits: +- Small increments +- When adding features we can spot regressions quickly due to having numerous tests +- Aggressive refactoring possible due to public API being tested +- Complete regression test suite
\ No newline at end of file diff --git a/Device mapper.md b/Device mapper.md new file mode 100644 index 0000000..150338b --- /dev/null +++ b/Device mapper.md @@ -0,0 +1,30 @@ +--- +tags: + - linux + - devices + - kernel +--- +Device mapper is a kernel driver to map physical block devices to virtual ones; this means that for example we can have many physical devices be represented by a single virtual one (linear mapping). + +How does it work? +What is its function in relation to multipath? + +Logical devices using device-mapper can be managed using `man 8 dmsetup` +``` +[jasras@n04.compute.vps2-lej1 ~]$ sudo dmsetup info 3600a098038314d736724566a67346538 +Name: 3600a098038314d736724566a67346538 +State: ACTIVE +Read Ahead: 256 +Tables present: LIVE +Open count: 1 +Event number: 3 +Major, minor: 253, 12 +Number of targets: 1 +UUID: mpath-3600a098038314d736724566a67346538 +``` + +If dmsetup cannot remove a device because a process still has it open, but lsof does not show any processes that open it, use `-f` which replaces the device with a fake that rejects all I/O. + +--- +[device-mapper](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/device_mapper#device_mapper) +[dmsetup](https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/dmsetup#dmsetup)
\ No newline at end of file diff --git a/Discovery Testing.md b/Discovery Testing.md new file mode 100644 index 0000000..9495dfe --- /dev/null +++ b/Discovery Testing.md @@ -0,0 +1,8 @@ +--- +tags: + - tdd + - architecture + - software-design + - software-engineering +--- +Is a form of [[TDD]]
\ No newline at end of file diff --git a/High Gear Low Gear Testing.md b/High Gear Low Gear Testing.md new file mode 100644 index 0000000..a33d223 --- /dev/null +++ b/High Gear Low Gear Testing.md @@ -0,0 +1,14 @@ +--- +tags: + - software-design + - software-engineering + - tdd +--- +This phrase came from the https://www.cosmicpython.com/ book.. The core idea here is that during development you write **many** small unit-tests that make it easy and fast to test whether we are doing the correct thing. +Having many such small tests makes it hard to refactor, so at some point we can start dropping some of the very specific tests when we have more general tests and feel confident that everything works as intended. This makes it less brittle, because small very specific tests will start failing the moment you start refactoring a little bit. + +Then when you need to start developing again you start by writing new very specific tests until you again feel confident that everything works as intended and is also tested by more general tests. + +The analog is a bit to cycling: when you start to move you set the bicycle in a low gear and then once you are going you shift the gear lower and lower until you are at speed and comfortable. + +In my mind it is still a bit unclear when that inflection point occurs that you decide to drop specific tests though, but I guess that also kind of comes with practice and experience.
\ No newline at end of file diff --git a/How to (RIP) Grep.md b/How to (RIP) Grep.md new file mode 100644 index 0000000..f7122a3 --- /dev/null +++ b/How to (RIP) Grep.md @@ -0,0 +1,17 @@ +--- +tags: + - howto + - grep +--- + +`grep -vxf gobs_servers onecrm_servers` grep all onecrm_servers that **ARE NOT** in gobs_servers +`grep -oxf gobs_servers onecrm_servers` grep all onecrm_servers that **ARE** in gobs_servers +`grep -vxf onecrm_servers gobs_servers` grep all gobs_servers that **ARE NOT** in onecrm_servers +`grep -oxf onecrm_servers gobs_servers` grep all gobs_servers that **ARE** in onecrm_servers + + +# Useful Ripgrep flags + + `-L` to make it follow symlinks: this is handy for example when searching hieradata, because node definitions can be symlinked. + +`-l` to only print filenames that contain matches.
\ No newline at end of file diff --git a/List of tags I use in this Vault and their purpose.md b/List of tags I use in this Vault and their purpose.md new file mode 100644 index 0000000..1b514d3 --- /dev/null +++ b/List of tags I use in this Vault and their purpose.md @@ -0,0 +1,17 @@ +--- +tags: + - self + - reflection + - howto +--- +I divide tags into seperate categories, they are as follows: + +# Functional tags +All tags ultimately are there to categorize content, but these tags indicate the note has a specific function or goal. + +- **self**: notes that are about me +- **reflection**: notes that contain some wisdom from looking in the mirror and seeing myself, they often go together with `self` +- **howto**: a note on how to do something, such as this one, how to use tags + +# Category tags +These are just to categorize notes based on for example subject.
\ No newline at end of file diff --git a/London-style TDD.md b/London-style TDD.md new file mode 100644 index 0000000..a3c29c2 --- /dev/null +++ b/London-style TDD.md @@ -0,0 +1,8 @@ +--- +tags: + - tdd + - software-design + - software-engineering +--- +Is very similar to [[Detroit-style TDD]] but there are some differences. +[[London-style versus Detroit-style TDD]]
\ No newline at end of file diff --git a/London-style versus Detroit-style TDD.md b/London-style versus Detroit-style TDD.md new file mode 100644 index 0000000..7c10d4d --- /dev/null +++ b/London-style versus Detroit-style TDD.md @@ -0,0 +1,12 @@ +--- +tags: + - tdd + - software-design + - software-engineering +--- +Detroit-style TDD focuses mainly on working bottom up, starting with the core of the domain working outward. +London-style TDD works outside in, starting at an API endpoint or controller working inwards. + +The primary con of london style is to build things that aren't needed ([[YAGNI]]) + +https://github.com/testdouble/contributing-tests/wiki/London-school-TDD
\ No newline at end of file diff --git a/Managing Ceph on Devstack & OpenStack.md b/Managing Ceph on Devstack & OpenStack.md index cad83e5..6ccb1c5 100644 --- a/Managing Ceph on Devstack & OpenStack.md +++ b/Managing Ceph on Devstack & OpenStack.md @@ -39,4 +39,14 @@ rbd image 'volume-6f5e81dd-40f3-42f9-aad3-afcf5696387c': modify_timestamp: Mon Mar 31 12:56:46 2025 parent: volumes/volume-5b6cc251-8f3e-4572-a8b9-52efa390ebc3@snapshot-3d1f148a-5fe4-4df2-87c3-4272565b82c2 overlap: 10 GiB +``` + +using `rbd du` we can get some actual usage stats: +``` +ubuntu@gobs-devstack:~$ sudo rbd du volumes/volume-d66216fc-0b4a-4480-be0e-5530fb8e6004 +NAME PROVISIONED USED +volume-d66216fc-0b4a-4480-be0e-5530fb8e6004@volume-e65e7bcf-88a8-4afb-ba5c-0dbb27f402d9.clone_snap 1 GiB 44 MiB +volume-d66216fc-0b4a-4480-be0e-5530fb8e6004@snapshot-05b07e40-8cbc-4f2d-896c-bbfc0cd2c801 10 GiB 52 MiB +volume-d66216fc-0b4a-4480-be0e-5530fb8e6004 10 GiB 10 GiB +<TOTAL> 10 GiB 10 GiB ```
\ No newline at end of file diff --git a/OpenStack Tempest.md b/OpenStack Tempest.md new file mode 100644 index 0000000..d7d84c4 --- /dev/null +++ b/OpenStack Tempest.md @@ -0,0 +1,7 @@ +--- +tags: + - tempest + - openstack +--- +Test an OpenStack cloud as if it is an end-user. +**Scenario Test** is more complicated than a single APi endpoint, like, booting from an existing volume.
\ No newline at end of file @@ -0,0 +1,12 @@ +--- +tags: + - tdd + - software-design + - software-engineering +--- +Test Driven Development (or Design) is a software engineering paradigm where one starts off with writing a failing test (suite) and from there tries to implement code to ultimately make the test succeed. + +It is mainly divided into three styles: +- [[Detroit-style TDD]] +- [[London-style TDD]] +- [[Discovery Testing]]
\ No newline at end of file diff --git a/Tempest Plugin.md b/Tempest Plugin.md new file mode 100644 index 0000000..58ebaac --- /dev/null +++ b/Tempest Plugin.md @@ -0,0 +1,18 @@ +--- +tags: + - openstack + - tempest +--- +A plugin for [[OpenStack Tempest]] that tests a non-core service such as gobs. + +**Recommendation:** keep tempest plugin for a service in a seperate repository. + +Requires an entrypoint in your app that it can find. +Consists of "service clients", clients that consume specific parts of the service. ie. a backup service client, or snapshot service client. + +1 test suite per service / resource of the API. + +Base class for "API Tests" can handle resource management using an idempotent id it can make sure resources are properly destroyed upon teardown. + +--- +https://www.youtube.com/watch?v=oN4lRR2_S_k
\ No newline at end of file diff --git a/The volume type of a Cinder volume can be changed.md b/The volume type of a Cinder volume can be changed.md new file mode 100644 index 0000000..8ee52ad --- /dev/null +++ b/The volume type of a Cinder volume can be changed.md @@ -0,0 +1,7 @@ +--- +tags: + - openstack + - cinder + - howto +--- +Shelve server, set new volume type, unshelve server.
\ No newline at end of file diff --git a/Writing a test double before the real implementation gives feedback on design.md b/Writing a test double before the real implementation gives feedback on design.md new file mode 100644 index 0000000..9412676 --- /dev/null +++ b/Writing a test double before the real implementation gives feedback on design.md @@ -0,0 +1,14 @@ +--- +tags: + - tdd + - software-design + - architecture + - software-engineering +--- +https://github.com/testdouble/contributing-tests/wiki/Don't-mock-what-you-don't-own + +The primary value of a test double from a TDD perspective is that it provides **cheap feedback** on the design of an object. Because we can play with its interface before actually having to use it, it is cheap to throw away and start over (vs having implemented all sorts of other stuff around it). + +This is alternatively called "Increased Usage", referring to that a test is an additional usage of a function (which normally sometimes have just a single caller anyway) and that while writing a few test cases you'll quickly get a feel of if the API is good or bad. + +[[London-style versus Detroit-style TDD]]
\ No newline at end of file diff --git a/YAGNI.md b/YAGNI.md new file mode 100644 index 0000000..cc43d7a --- /dev/null +++ b/YAGNI.md @@ -0,0 +1,8 @@ +--- +tags: + - tdd + - software-engineering + - software-design + - architecture +--- +It means "You Ain't Gonna Need It". It refers to engineers often building stuff that isn't actually going to be used.
\ No newline at end of file diff --git a/daily/10-Apr-2025.md b/daily/10-Apr-2025.md new file mode 100644 index 0000000..8ac3187 --- /dev/null +++ b/daily/10-Apr-2025.md @@ -0,0 +1,18 @@ +--- +tags: + - self + - reflection +--- +### I assumed that Mohammed made an oopsie but instead it turned out to be one of us that forgot to clean up. +Today I found out that on the testpod the user sanoid's ssh keys suddenly belonged to Mohammeds' user. I right away went to his chat with the idea that he did something silly probably and even told him that it made me a bit worried. + +I also went to Rutger, who immediately pointed out that it is probably due to us changing the uid of sanoids' user. Which turned out to be true. + +I ask myself the following: why am I quick to jump to a conclusion like "Ah, mohammed might've caused damage by accidentally chowning to much or something like that." +How can I stop myself from that? +I think one way is to ALWAYS force myself to investigate completely, not right away talking about it with others. Keep it to myself, until I really MUST communicate about it. + + +#### Erik doesn't show up at the office for Carlos even though he said that he would be there on Thursday +This kind of triggers a feeling that Erik has a bit of a lax attitude, which often was associated with ops back in the day as well. +He is the designated mentor but I feel he doesn't prepare it well and just "goes with the flow" too much.
\ No newline at end of file diff --git a/daily/11-Apr-2025.md b/daily/11-Apr-2025.md new file mode 100644 index 0000000..8b0a2cf --- /dev/null +++ b/daily/11-Apr-2025.md @@ -0,0 +1,27 @@ +--- +tags: + - weekly +--- +This week: +- [[10-Apr-2025]] + +Today marks another Friday, almost weekend, woohoo. + +This week few notable things happened. + +First of all I finally have restored a Ceph backed volume successfully! The issue was actually kind of silly, I forgot to close the read end of a pipe, thus it kept blocking. Luckily I found it, and fixing it was rather trivial. +After that I refactored a bit, to make it little bit better (still not great), and deployed. +I also fixed the request ID logging that was broken for a while, during refactoring of the agent RPC handler I accidentally moved the ctx.update_store outside of the child thread, so the update was useless. Moving it back into the child thread gave back our precious request ids. +Oh and I also found the cause of some sporadic mysql "object belongs to a different session" issue in the backup service that was haunting me. This happened due the Unit Of Work being instantiated only during application startup, specifically for the RPC handler, and then every RPC request used that same UOW. +Because the UOW creates a new session everytime it wasn't completely broken, but occasionally two RPC calls could come in at the same time and then the latter would override the session of the first. +I fixed this by instantiating a UOW per request, this also happens in the API, and is actually the correct way of using it. + +Then I have kicked off the [[List of tags I use in this Vault and their purpose]] note, which contains a list of tags I use within this vault so I don't forget. +As with the current note i'm writing I added the new "weekly" tag to indicate that this "daily" note is actually a week report which I want to write every friday from now on. + +Just had a little brainfart writing the above.. Is it too long? I plan to use these weekly notes to introspect during self assessments, but of course they shouldn't be too tedious to go through... Hmm, well, I guess we'll have to actually **use** it before deciding that. + +I think it will be good practice to link to the current weeks' notes in this weekly note as well so.. see the top :) I probably should put more stuff into daily notes, and then make this a bit of a TLDR. + +I have also been doing some thinking and note taking about [[TDD]] because I feel kind of bad about the current state of the backup service & agent (no tests). We've been looking at [[OpenStack Tempest]] for a bit which is interesting, but I would also like to just create better and more unit tests, especially during development. [[High Gear Low Gear Testing]] was a phrase from the cosmic python book that particularly seemed to resonate with me, but I haven't yet been able to practice it. + |