From 80153a133d0888b7c0a7b72a9f80ccc6f005c555 Mon Sep 17 00:00:00 2001 From: Jasper Ras Date: Sat, 23 Nov 2024 17:34:50 +0100 Subject: sorted out inbox --- .obsidian/workspace.json | 66 ++++---- .trash/2024-01-08.md | 16 ++ .trash/2024-01-09.md | 24 +++ .trash/2024-01-10.md | 5 + .trash/2024-02-20.md | 12 ++ .trash/2024-06-13.md | 2 + .trash/2024-06-17.md | 10 ++ .trash/2024-06-26.md | 0 .trash/2024-08-21.md | 20 +++ .trash/2024-08-22.md | 4 + .trash/2024-08-23.md | 6 + .trash/2024-08-26.md | 1 + .trash/2024-08-27.md | 11 ++ .trash/2024-08-28.md | 1 + .trash/2024-08-29.md | 0 .trash/2024-09-03.md | 1 + .trash/2024-09-10.md | 1 + .trash/2024-09-30.md | 1 + .trash/2024-10-02.md | 6 + .trash/2024-10-04.md | 26 ++++ .trash/2024-10-15.md | 3 + .trash/2024-11-11.md | 3 + .trash/2024-11-12.md | 3 + .trash/ELF (files).md | 0 .trash/Group.ONE Backup Service.md | 6 + .trash/Neutron DNS & Designate.md | 3 + .trash/Nix Flakes.md | 5 + .trash/Nixpkgs Overlays.md | 0 .trash/Nova Scheduler.md | 2 + .trash/Ruby gems nixpkgs.md | 0 .trash/Shell.nix python.md | 0 .trash/Timings VPS.md | 4 + .trash/Updates ansible ad-hoc.md | 0 .trash/Zoner Migration.md | 1 + .trash/elixir - modules.md | 0 .trash/intro.md | 7 + .trash/para.md | 2 + 0 inbox/2024-01-08.md | 16 -- 0 inbox/2024-01-09.md | 24 --- 0 inbox/2024-01-10.md | 5 - 0 inbox/2024-02-20.md | 12 -- 0 inbox/2024-06-13.md | 2 - 0 inbox/2024-06-17.md | 10 -- 0 inbox/2024-06-26.md | 0 0 inbox/2024-08-21.md | 27 ---- 0 inbox/2024-08-22.md | 25 --- 0 inbox/2024-08-23.md | 6 - 0 inbox/2024-08-26.md | 1 - 0 inbox/2024-08-27.md | 11 -- 0 inbox/2024-08-28.md | 1 - 0 inbox/2024-08-29.md | 0 0 inbox/2024-09-03.md | 1 - 0 inbox/2024-09-10.md | 1 - 0 inbox/2024-09-30.md | 1 - 0 inbox/2024-10-02.md | 6 - 0 inbox/2024-10-04.md | 26 ---- 0 inbox/2024-10-15.md | 3 - 0 inbox/2024-11-11.md | 3 - 0 inbox/2024-11-12.md | 3 - 0 inbox/Blkio errors.md | 14 -- 0 inbox/ELF (files).md | 0 0 inbox/Group.ONE Backup Service.md | 6 - 0 inbox/Issues met TC oplossen.md | 25 --- 0 inbox/Neutron DNS & Designate.md | 3 - 0 inbox/Nix Flakes.md | 21 --- 0 inbox/Nixpkgs Overlays.md | 0 0 inbox/Nova Scheduler.md | 2 - 0 inbox/Ruby gems nixpkgs.md | 0 0 inbox/Shell.nix python.md | 15 -- 0 inbox/Timings VPS.md | 4 - 0 inbox/Updates ansible ad-hoc.md | 19 --- 0 inbox/Zoner Migration.md | 1 - 0 inbox/antagonist_migratie.md | 27 ---- 0 inbox/intro.md | 21 --- 0 inbox/para.md | 10 -- 3 resources/Group.ONE Infrastructure.md | 1 - 3 resources/Man pages.md | 3 - 3 resources/Note taking.md | 7 - 3 resources/RabbitMQ.md | 2 - 3 resources/TaskWarrior (ToDo).md | 3 - 3 resources/ansible.md | 20 +++ 3 resources/elixir.md | 228 ++++++++++++++++++++++++++++ 3 resources/group.one infra.md | 1 + 3 resources/man pages.md | 2 + 3 resources/mermaidjs.md | 21 +++ 3 resources/nix.md | 58 +++++++ 3 resources/note taking system.md | 21 +++ 3 resources/programming/Elixir - modules.md | 0 3 resources/programming/Elixir.md | 228 ---------------------------- 3 resources/programming/Nix.md | 16 -- 3 resources/rabbitmq.md | 1 + 3 resources/taskwarrior.md | 4 + 4 archive/Blkio errors.md | 14 ++ 4 archive/Issues met TC oplossen.md | 25 +++ 4 archive/antagonist_migratie.md | 27 ++++ TODO.md | 1 + 96 files changed, 642 insertions(+), 645 deletions(-) create mode 100644 .trash/2024-01-08.md create mode 100644 .trash/2024-01-09.md create mode 100644 .trash/2024-01-10.md create mode 100644 .trash/2024-02-20.md create mode 100644 .trash/2024-06-13.md create mode 100644 .trash/2024-06-17.md create mode 100644 .trash/2024-06-26.md create mode 100644 .trash/2024-08-21.md create mode 100644 .trash/2024-08-22.md create mode 100644 .trash/2024-08-23.md create mode 100644 .trash/2024-08-26.md create mode 100644 .trash/2024-08-27.md create mode 100644 .trash/2024-08-28.md create mode 100644 .trash/2024-08-29.md create mode 100644 .trash/2024-09-03.md create mode 100644 .trash/2024-09-10.md create mode 100644 .trash/2024-09-30.md create mode 100644 .trash/2024-10-02.md create mode 100644 .trash/2024-10-04.md create mode 100644 .trash/2024-10-15.md create mode 100644 .trash/2024-11-11.md create mode 100644 .trash/2024-11-12.md create mode 100644 .trash/ELF (files).md create mode 100644 .trash/Group.ONE Backup Service.md create mode 100644 .trash/Neutron DNS & Designate.md create mode 100644 .trash/Nix Flakes.md create mode 100644 .trash/Nixpkgs Overlays.md create mode 100644 .trash/Nova Scheduler.md create mode 100644 .trash/Ruby gems nixpkgs.md create mode 100644 .trash/Shell.nix python.md create mode 100644 .trash/Timings VPS.md create mode 100644 .trash/Updates ansible ad-hoc.md create mode 100644 .trash/Zoner Migration.md create mode 100644 .trash/elixir - modules.md create mode 100644 .trash/intro.md create mode 100644 .trash/para.md delete mode 100644 0 inbox/2024-01-08.md delete mode 100644 0 inbox/2024-01-09.md delete mode 100644 0 inbox/2024-01-10.md delete mode 100644 0 inbox/2024-02-20.md delete mode 100644 0 inbox/2024-06-13.md delete mode 100644 0 inbox/2024-06-17.md delete mode 100644 0 inbox/2024-06-26.md delete mode 100644 0 inbox/2024-08-21.md delete mode 100644 0 inbox/2024-08-22.md delete mode 100644 0 inbox/2024-08-23.md delete mode 100644 0 inbox/2024-08-26.md delete mode 100644 0 inbox/2024-08-27.md delete mode 100644 0 inbox/2024-08-28.md delete mode 100644 0 inbox/2024-08-29.md delete mode 100644 0 inbox/2024-09-03.md delete mode 100644 0 inbox/2024-09-10.md delete mode 100644 0 inbox/2024-09-30.md delete mode 100644 0 inbox/2024-10-02.md delete mode 100644 0 inbox/2024-10-04.md delete mode 100644 0 inbox/2024-10-15.md delete mode 100644 0 inbox/2024-11-11.md delete mode 100644 0 inbox/2024-11-12.md delete mode 100644 0 inbox/Blkio errors.md delete mode 100644 0 inbox/ELF (files).md delete mode 100644 0 inbox/Group.ONE Backup Service.md delete mode 100644 0 inbox/Issues met TC oplossen.md delete mode 100644 0 inbox/Neutron DNS & Designate.md delete mode 100644 0 inbox/Nix Flakes.md delete mode 100644 0 inbox/Nixpkgs Overlays.md delete mode 100644 0 inbox/Nova Scheduler.md delete mode 100644 0 inbox/Ruby gems nixpkgs.md delete mode 100644 0 inbox/Shell.nix python.md delete mode 100644 0 inbox/Timings VPS.md delete mode 100644 0 inbox/Updates ansible ad-hoc.md delete mode 100644 0 inbox/Zoner Migration.md delete mode 100644 0 inbox/antagonist_migratie.md delete mode 100644 0 inbox/intro.md delete mode 100644 0 inbox/para.md delete mode 100644 3 resources/Group.ONE Infrastructure.md delete mode 100644 3 resources/Man pages.md delete mode 100644 3 resources/Note taking.md delete mode 100644 3 resources/RabbitMQ.md delete mode 100644 3 resources/TaskWarrior (ToDo).md create mode 100644 3 resources/ansible.md create mode 100644 3 resources/elixir.md create mode 100644 3 resources/group.one infra.md create mode 100644 3 resources/man pages.md create mode 100644 3 resources/mermaidjs.md create mode 100644 3 resources/nix.md create mode 100644 3 resources/note taking system.md delete mode 100644 3 resources/programming/Elixir - modules.md delete mode 100644 3 resources/programming/Elixir.md delete mode 100644 3 resources/programming/Nix.md create mode 100644 3 resources/rabbitmq.md create mode 100644 3 resources/taskwarrior.md create mode 100644 4 archive/Blkio errors.md create mode 100644 4 archive/Issues met TC oplossen.md create mode 100644 4 archive/antagonist_migratie.md create mode 100644 TODO.md diff --git a/.obsidian/workspace.json b/.obsidian/workspace.json index 4e0aa38..bf37445 100644 --- a/.obsidian/workspace.json +++ b/.obsidian/workspace.json @@ -4,21 +4,21 @@ "type": "split", "children": [ { - "id": "7d88ea66e5554b07", + "id": "27a70fc53d7bc8b7", "type": "tabs", "children": [ { - "id": "b4f55b03d2399001", + "id": "7e2bc46d82cc87a1", "type": "leaf", "state": { "type": "markdown", "state": { - "file": "Interviews/Interview Mohammed Al-Mahdawi.md", + "file": "3 resources/ansible.md", "mode": "source", "source": false }, "icon": "lucide-file", - "title": "Interview Mohammed Al-Mahdawi" + "title": "ansible" } } ] @@ -139,10 +139,10 @@ "state": { "type": "outline", "state": { - "file": "Interviews/Interview Mohammed Al-Mahdawi.md" + "file": "3 resources/ansible.md" }, "icon": "lucide-list", - "title": "Outline of Interview Mohammed Al-Mahdawi" + "title": "Outline of ansible" } } ], @@ -162,35 +162,35 @@ "command-palette:Open command palette": false } }, - "active": "0c5f7b9f79a1fa84", + "active": "7e2bc46d82cc87a1", "lastOpenFiles": [ - "Interviews/Interview Mohammed Al-Mahdawi.md", - "Interviews/Vragenlijst.md", - "Interviews/Interview Patrick Spek.md", + "0 inbox/Zoner Migration.md", + "3 resources/ansible.md", + "0 inbox/Updates ansible ad-hoc.md", + "3 resources/nix.md", + "0 inbox/Timings VPS.md", + "0 inbox/Shell.nix python.md", + "TODO.md", + "0 inbox/Ruby gems nixpkgs.md", + "0 inbox/para.md", + "0 inbox/Nova Scheduler.md", + "3 resources/mermaidjs.md", + "0 inbox/Nixpkgs Overlays.md", + "0 inbox/Nix Flakes.md", + "zettelkast/Index/Nix.md", + "4 archive/Issues met TC oplossen.md", + "0 inbox/Neutron DNS & Designate.md", + "3 resources/note taking system.md", + "0 inbox/intro.md", + "4 archive/Blkio errors.md", + "0 inbox/Group.ONE Backup Service.md", + "0 inbox/ELF (files).md", + "4 archive/antagonist_migratie.md", + "0 inbox/2024-11-12.md", + "0 inbox/2024-11-11.md", + "0 inbox/2024-10-15.md", + "0 inbox/2024-10-04.md", "Interviews", - "2024-11-11.md", - "2024-11-12.md", - "2024-10-15.md", - "2024-10-04.md", - "Daily/2024-10-02.md", - "Daily/2024-08-23.md", - "Daily/2024-08-27.md", - "Daily/2024-08-29.md", - "Daily/2024-09-03.md", - "Daily/2024-09-10.md", - "Daily/2024-09-30.md", - "Refined/2024-10-03.md", - "Daily/2024-08-28.md", - "Daily/2024-08-26.md", - "Daily/2024-08-22.md", - "Daily/2024-08-21.md", - "Elixir.md", - "Elixir - modules.md", - "Refined/RabbitMQ.md", - "Refined/TaskWarrior (ToDo).md", - "Refined/Group.ONE Infrastructure.md", - "Refined/Man pages.md", - "Refined/Note taking.md", "Refined", "Daily", "Untitled.canvas", diff --git a/.trash/2024-01-08.md b/.trash/2024-01-08.md new file mode 100644 index 0000000..7e480af --- /dev/null +++ b/.trash/2024-01-08.md @@ -0,0 +1,16 @@ +--- +tags: + - qemu + - qemu-guest-agent + - backups +--- +ot QEMU Guest agent working via booting a VM with an image after setting `hw_qemu_guest_agent='yes'` on that image. + +Put a script under `/etc/qemu/fsfreeze-hook.d/` with the executable flag set that echo's 'yo' when it is run. + +Tried Cinder backup: script not called +Tried Cinder snapshot: script not called +Tried Nova snapshot: script not called. + +--- +10.40 - 16.00 \ No newline at end of file diff --git a/.trash/2024-01-09.md b/.trash/2024-01-09.md new file mode 100644 index 0000000..9b3c874 --- /dev/null +++ b/.trash/2024-01-09.md @@ -0,0 +1,24 @@ +--- +tags: + - idea + - oslo_messaging + - tuning + - maintenance + - issue +--- +ad an idea to develop a tool to parse openstack release pages to see what's new from the CLI. + +Continued maintenance after making sure cold migrations do work on running VMs and make sure they are started again after migrating. + +Messaging timeouts occurred during maintenance. Found some settings to tune the neutron and nova oslo messaging library. +https://docs.mirantis.com/mcp/q4-18/mcp-deployment-guide/advanced-config/tune-rabbitmq-perf.html + +Learned about the HAProxy ansible module: +https://docs.ansible.com/ansible/latest/collections/community/general/haproxy_module.html + +We decided to add ansible playbooks for enabling and disabling services. So we can disable the neutron controller in haproxy before restarting it. This prevents an issue where it is being flooded with requests while starting up and causing it to process SSL requests before even having loaded the SSL certificates. + +Forgot that I had to visit the mortgage advisor at the end of the afternoon. + +--- +9.30 - 16.00 diff --git a/.trash/2024-01-10.md b/.trash/2024-01-10.md new file mode 100644 index 0000000..6a91cdb --- /dev/null +++ b/.trash/2024-01-10.md @@ -0,0 +1,5 @@ +Maintenance had been stopped due to an hanging apt-lock on n40 vps2 compute. Killed the hanging process and resumed maintenance. + + +--- +9.30 \ No newline at end of file diff --git a/.trash/2024-02-20.md b/.trash/2024-02-20.md new file mode 100644 index 0000000..2359685 --- /dev/null +++ b/.trash/2024-02-20.md @@ -0,0 +1,12 @@ +Building an OpenStack service. + +**Things to look into** +- Tox (testing I think) +- Project structure +- Conventions +- Deployment considerations +- https://opendev.org/openstack/requirements/ + +**Tox** +https://tox.wiki/en/4.13.0/ +Test automation and standardization. \ No newline at end of file diff --git a/.trash/2024-06-13.md b/.trash/2024-06-13.md new file mode 100644 index 0000000..61a16ac --- /dev/null +++ b/.trash/2024-06-13.md @@ -0,0 +1,2 @@ +Tests: +- \ No newline at end of file diff --git a/.trash/2024-06-17.md b/.trash/2024-06-17.md new file mode 100644 index 0000000..144e133 --- /dev/null +++ b/.trash/2024-06-17.md @@ -0,0 +1,10 @@ +Investigated the last remaining error with a missing reply queue which was the likely suspect of some alerts of compute service down etc. After some digging in Kibana i noticed a compute node that was logging MessagingTimeouts. I restarted the compute service on this node and it seems to have resolved the errors. + +Both system test in CPH8 and LEJ1 were failing. CPH8 likely due to me restarting neutron-server earlier that morning in an attempt to fix the reply queue error. +LEJ1 had been failing since 14th of June. My guess is due to some maintenance related issue. Both system tests are running smoothly again after cleaning up their artifacts. + +Meanwhile, maintenance on VPS1-LEJ1 is still on-going and causing headaches. Another password refresh of the access node caused a migration to fail during post-cleanup and I had to set the host and node of the instance to the correct node directly via MySQL. I should apply Rutgers' suggestion of saving some variables in the OS_\* env vars, rather sooner than later. + +Tested retention backups being scheduled, and deployed it to prod. + +Investigate whether we can listen for notifications by Nova so that we can listen for server deletions and remove them from periodic backups. There should be no need to delete backups for the server, as the retention policy will have them deleted after a while. \ No newline at end of file diff --git a/.trash/2024-06-26.md b/.trash/2024-06-26.md new file mode 100644 index 0000000..e69de29 diff --git a/.trash/2024-08-21.md b/.trash/2024-08-21.md new file mode 100644 index 0000000..cf7a783 --- /dev/null +++ b/.trash/2024-08-21.md @@ -0,0 +1,20 @@ +eed to start learning to take notes consistently. My idea now, to kick-off this process, is to open a daily note in obsidian every day at the start of the day in which I will keep crude, quick notes. +Then at the end of the day I shall take some time to go over the notes and refine them. Not yet sure if that refinement will already be the end result or if i should go over refined notes again at maybe the end of each week to see if they really stick. +I should commit notes to git to save them. + + + +Don't forget to always read man pages. Maybe I should create a detailed note about how to use man pages. + +I will start going to the office full-time from now on. +I should make sure my Obsidian notes and taskwarrior tasks are kept safely in a git repo in case i lose my laptop.l + + + +I should ask for a diagram of the current systems and take note of it, e.g what is OneCRM / OneHOP / OneHome and all that. + +To upgrade a single package using apt we can use `apt-get install --only-upgrade `. + +Need to "relearn" to design stuff and document it. + +Designing structures and a mechanism to return and store info about individual disk backups, we can send that as parent when making incremental backups. Currently we fetch parent reference from checkpoint, which in case of volume's doesn't work of course. It also breaks if, on the testpod, we reset all backups etc but dont do so with all checkpoints. \ No newline at end of file diff --git a/.trash/2024-08-22.md b/.trash/2024-08-22.md new file mode 100644 index 0000000..a591d25 --- /dev/null +++ b/.trash/2024-08-22.md @@ -0,0 +1,4 @@ +Woke up full of motivation to go to the office. Asked my honey if it's okay to join her by train "is there enough time left?", "yeah" she replied. Then when I come downstairs and she asks me if I can be fast, I tell her maybe it's better you go then bc i worry I will not be able to. Right away she storms off mad and angry... My motivation gone, so I'll work from home.. + +Went to the office anyway by motorcycle, cannot feel like I'm waking up when staying at home anymore. + diff --git a/.trash/2024-08-23.md b/.trash/2024-08-23.md new file mode 100644 index 0000000..6f41ca2 --- /dev/null +++ b/.trash/2024-08-23.md @@ -0,0 +1,6 @@ +Discussed about Automatic Evac (HA). We were unsure about what to do when a local storage goes down. Do we restore the instances from backups to a new node? In that case a DB server might lose important transtactions. So we could also wait until it is fixed. + +Would like Rutger to explain the Freenet BGP load balancing setup in more detail. How do the routers work, what are all those interfaces attached to them? + +A local storage disk backup is stored as a QCOW2 image. An incremental has a relationship to its parent, because they neged to be rebased onto each other. +A volume disk backup is stored as raw data and is accompanied by a file of checksums per block. The extra file is used both to track dirtied blocks for incremental backup. \ No newline at end of file diff --git a/.trash/2024-08-26.md b/.trash/2024-08-26.md new file mode 100644 index 0000000..a91a1c9 --- /dev/null +++ b/.trash/2024-08-26.md @@ -0,0 +1 @@ +Nothing of note \ No newline at end of file diff --git a/.trash/2024-08-27.md b/.trash/2024-08-27.md new file mode 100644 index 0000000..eb75d3f --- /dev/null +++ b/.trash/2024-08-27.md @@ -0,0 +1,11 @@ +Changed a few things in the backup service: +* No longer delete all checkpoints when we make a full backup + * TODO: make sure we delete the checkpoint corresponding to a backup when the backup is deleted. + * TODO: Gracefully handle all checkpoints being deleted by openstack +* No longer explicitly specify "incremental" bool, just make it a best effort attempt. + * Still need to work out restores + + +When can we reboot n01.compute vps1-cph8. Its uptime is already 300 days. + +Short call with Jakub. Their priorities have shifted again, the focus on backup will be moved to improving performance of OneHome. They plan to synchronize data from OpenStack to OneHome so they can serve it faster. He asked me whether the customer can influence OpenStack state from within their VM. I told him that probably the only thing they can do to influence OpenStack is shutdown their VM. \ No newline at end of file diff --git a/.trash/2024-08-28.md b/.trash/2024-08-28.md new file mode 100644 index 0000000..ac0cac1 --- /dev/null +++ b/.trash/2024-08-28.md @@ -0,0 +1 @@ +Had a meeting about onehome performance, they query everything in existence for a customer because of their graphQL workflow. they want to now sync everything in existence to a table so their gQL is fast again. \ No newline at end of file diff --git a/.trash/2024-08-29.md b/.trash/2024-08-29.md new file mode 100644 index 0000000..e69de29 diff --git a/.trash/2024-09-03.md b/.trash/2024-09-03.md new file mode 100644 index 0000000..1d8a68e --- /dev/null +++ b/.trash/2024-09-03.md @@ -0,0 +1 @@ +one on one Rutger; diff --git a/.trash/2024-09-10.md b/.trash/2024-09-10.md new file mode 100644 index 0000000..07ff5ba --- /dev/null +++ b/.trash/2024-09-10.md @@ -0,0 +1 @@ +EYQBGNF95VORXXVVJ6NKFZF expressVPN \ No newline at end of file diff --git a/.trash/2024-09-30.md b/.trash/2024-09-30.md new file mode 100644 index 0000000..fef578b --- /dev/null +++ b/.trash/2024-09-30.md @@ -0,0 +1 @@ +- Check freenetvps gobs \ No newline at end of file diff --git a/.trash/2024-10-02.md b/.trash/2024-10-02.md new file mode 100644 index 0000000..64c611f --- /dev/null +++ b/.trash/2024-10-02.md @@ -0,0 +1,6 @@ +* Fixed n06 testpod, due to installing stuff via pip we had some conflicting packages being used. Pip has been purged and stuff seems to work normally now. +* Investigated DNS issues on testpod lb with Erik. +* Fixed some small issues in the backup service + * Deleted servers couldn't be removed from the periodic backup, that is now fixed. + * Less noisy logging by moving some stuff to debug level. +* Deployed image_id change to prod \ No newline at end of file diff --git a/.trash/2024-10-04.md b/.trash/2024-10-04.md new file mode 100644 index 0000000..9c77b37 --- /dev/null +++ b/.trash/2024-10-04.md @@ -0,0 +1,26 @@ +What is needed to recreate a deleted server: +* name +* description +* block device mapping +* image properties + * while restoring we need to somehow schedule according to image properties, e.g os_type, or is this already resolved by selecting the correct AZ? + * volumes need to clone volume_image_properties +* port mapping +* availability zone +* flavor +* propertieskjk + + + +port +* name +* description +* allowed_address_pairs +* dns_domain +* dns_name +* extra_dhcp_opts +* fixed_ips +* is_port_security_enabled +* security_group_ids +* qos_policy_id +* network_id \ No newline at end of file diff --git a/.trash/2024-10-15.md b/.trash/2024-10-15.md new file mode 100644 index 0000000..8887a01 --- /dev/null +++ b/.trash/2024-10-15.md @@ -0,0 +1,3 @@ +Reminder: +* write SOP for fix_allocation +* write SOP for fixing duplicate connector \ No newline at end of file diff --git a/.trash/2024-11-11.md b/.trash/2024-11-11.md new file mode 100644 index 0000000..5f3eedb --- /dev/null +++ b/.trash/2024-11-11.md @@ -0,0 +1,3 @@ +Sqlalchemy/mysql connections lijken een timeout te hitten +zie: +`https://kibana.one.com/s/openstack/app/discover#/?_g=(filters:!(),refreshInterval:(pause:!t,value:60000),time:(from:now-7d%2Fd,to:now))&_a=(columns:!(),dataSource:(dataViewId:d80a7401-b1fd-43b2-bdfc-79dffb0e8b15,type:dataView),filters:!(),hideChart:!f,interval:auto,query:(language:kuery,query:%22cb3204fa-208f-4b90-95a0-846a9e13852b%22),sort:!(!('@timestamp',desc)))` diff --git a/.trash/2024-11-12.md b/.trash/2024-11-12.md new file mode 100644 index 0000000..e48d53f --- /dev/null +++ b/.trash/2024-11-12.md @@ -0,0 +1,3 @@ +Prometheus: +- scrape config: `roles/prometheus/onecom-prometheus-scrape-config-*.json` +- \ No newline at end of file diff --git a/.trash/ELF (files).md b/.trash/ELF (files).md new file mode 100644 index 0000000..e69de29 diff --git a/.trash/Group.ONE Backup Service.md b/.trash/Group.ONE Backup Service.md new file mode 100644 index 0000000..ebb2c92 --- /dev/null +++ b/.trash/Group.ONE Backup Service.md @@ -0,0 +1,6 @@ +# Ubiquitous language +**Project** The customer or tenant that owns the servers, backups and periodic backups. +**Server** Is the subject of a backup or periodic backup. It has one or more disks assigned. +**Disk** Can either be a local storage disk or shared storage volume. It is attached to a server and can be referenced by its name. +**Backup** Is a snapshot in time of a server. It can be used to restore a server to how it was exactly at that moment in time. +**PeriodicBackup** Is a group of servers that will be backed up periodically. \ No newline at end of file diff --git a/.trash/Neutron DNS & Designate.md b/.trash/Neutron DNS & Designate.md new file mode 100644 index 0000000..48785ab --- /dev/null +++ b/.trash/Neutron DNS & Designate.md @@ -0,0 +1,3 @@ +https://docs.openstack.org/neutron/latest/admin/config-dns-int.html#config-dns-int + +Lijkt er op dat dns_domain overruled kan worden op port niveau. \ No newline at end of file diff --git a/.trash/Nix Flakes.md b/.trash/Nix Flakes.md new file mode 100644 index 0000000..212281e --- /dev/null +++ b/.trash/Nix Flakes.md @@ -0,0 +1,5 @@ +nix + + + +--- diff --git a/.trash/Nixpkgs Overlays.md b/.trash/Nixpkgs Overlays.md new file mode 100644 index 0000000..e69de29 diff --git a/.trash/Nova Scheduler.md b/.trash/Nova Scheduler.md new file mode 100644 index 0000000..1f58400 --- /dev/null +++ b/.trash/Nova Scheduler.md @@ -0,0 +1,2 @@ +Applies Filters to filters compute hosts that are not usable until a list remains of usable compute hosts. +Filters have different costs, thus it's a good idea to put lightweight filters first as to reduce the load on the heavyweight ones. \ No newline at end of file diff --git a/.trash/Ruby gems nixpkgs.md b/.trash/Ruby gems nixpkgs.md new file mode 100644 index 0000000..e69de29 diff --git a/.trash/Shell.nix python.md b/.trash/Shell.nix python.md new file mode 100644 index 0000000..e69de29 diff --git a/.trash/Timings VPS.md b/.trash/Timings VPS.md new file mode 100644 index 0000000..51857df --- /dev/null +++ b/.trash/Timings VPS.md @@ -0,0 +1,4 @@ +10 VPS: avg ms over 100 reqs: 225 ms +20 VPS: first request: 247 ms. avg ms over 100 reqs: 203 ms +50 VPS: first request: 259 ms. avg ms over 100 reqs: 220 ms +101 VPS: first request: 264 ms. avg ms over 100 reqs: 221 ms diff --git a/.trash/Updates ansible ad-hoc.md b/.trash/Updates ansible ad-hoc.md new file mode 100644 index 0000000..e69de29 diff --git a/.trash/Zoner Migration.md b/.trash/Zoner Migration.md new file mode 100644 index 0000000..dc61370 --- /dev/null +++ b/.trash/Zoner Migration.md @@ -0,0 +1 @@ +* Kernel installation \ No newline at end of file diff --git a/.trash/elixir - modules.md b/.trash/elixir - modules.md new file mode 100644 index 0000000..e69de29 diff --git a/.trash/intro.md b/.trash/intro.md new file mode 100644 index 0000000..5204005 --- /dev/null +++ b/.trash/intro.md @@ -0,0 +1,7 @@ +# Foundational document +This document outlines the basis of my note taking system and serves mostly as a reminder to myself. + +I want to train the skill of effective note taking as I believe that it will serve as a foundation +to any other study and will boost my overall productivity. + + diff --git a/.trash/para.md b/.trash/para.md new file mode 100644 index 0000000..c44b5b7 --- /dev/null +++ b/.trash/para.md @@ -0,0 +1,2 @@ +# PARA method + diff --git a/0 inbox/2024-01-08.md b/0 inbox/2024-01-08.md deleted file mode 100644 index 13eee22..0000000 --- a/0 inbox/2024-01-08.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -tags: - - qemu - - qemu-guest-agent - - backups ---- -Got QEMU Guest agent working via booting a VM with an image after setting `hw_qemu_guest_agent='yes'` on that image. - -Put a script under `/etc/qemu/fsfreeze-hook.d/` with the executable flag set that echo's 'yo' when it is run. - -Tried Cinder backup: script not called -Tried Cinder snapshot: script not called -Tried Nova snapshot: script not called. - ---- -10.40 - 16.00 \ No newline at end of file diff --git a/0 inbox/2024-01-09.md b/0 inbox/2024-01-09.md deleted file mode 100644 index e0d0937..0000000 --- a/0 inbox/2024-01-09.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -tags: - - idea - - oslo_messaging - - tuning - - maintenance - - issue ---- -Had an idea to develop a tool to parse openstack release pages to see what's new from the CLI. - -Continued maintenance after making sure cold migrations do work on running VMs and make sure they are started again after migrating. - -Messaging timeouts occurred during maintenance. Found some settings to tune the neutron and nova oslo messaging library. -https://docs.mirantis.com/mcp/q4-18/mcp-deployment-guide/advanced-config/tune-rabbitmq-perf.html - -Learned about the HAProxy ansible module: -https://docs.ansible.com/ansible/latest/collections/community/general/haproxy_module.html - -We decided to add ansible playbooks for enabling and disabling services. So we can disable the neutron controller in haproxy before restarting it. This prevents an issue where it is being flooded with requests while starting up and causing it to process SSL requests before even having loaded the SSL certificates. - -Forgot that I had to visit the mortgage advisor at the end of the afternoon. - ---- -9.30 - 16.00 diff --git a/0 inbox/2024-01-10.md b/0 inbox/2024-01-10.md deleted file mode 100644 index 6a91cdb..0000000 --- a/0 inbox/2024-01-10.md +++ /dev/null @@ -1,5 +0,0 @@ -Maintenance had been stopped due to an hanging apt-lock on n40 vps2 compute. Killed the hanging process and resumed maintenance. - - ---- -9.30 \ No newline at end of file diff --git a/0 inbox/2024-02-20.md b/0 inbox/2024-02-20.md deleted file mode 100644 index 2359685..0000000 --- a/0 inbox/2024-02-20.md +++ /dev/null @@ -1,12 +0,0 @@ -Building an OpenStack service. - -**Things to look into** -- Tox (testing I think) -- Project structure -- Conventions -- Deployment considerations -- https://opendev.org/openstack/requirements/ - -**Tox** -https://tox.wiki/en/4.13.0/ -Test automation and standardization. \ No newline at end of file diff --git a/0 inbox/2024-06-13.md b/0 inbox/2024-06-13.md deleted file mode 100644 index 61a16ac..0000000 --- a/0 inbox/2024-06-13.md +++ /dev/null @@ -1,2 +0,0 @@ -Tests: -- \ No newline at end of file diff --git a/0 inbox/2024-06-17.md b/0 inbox/2024-06-17.md deleted file mode 100644 index 144e133..0000000 --- a/0 inbox/2024-06-17.md +++ /dev/null @@ -1,10 +0,0 @@ -Investigated the last remaining error with a missing reply queue which was the likely suspect of some alerts of compute service down etc. After some digging in Kibana i noticed a compute node that was logging MessagingTimeouts. I restarted the compute service on this node and it seems to have resolved the errors. - -Both system test in CPH8 and LEJ1 were failing. CPH8 likely due to me restarting neutron-server earlier that morning in an attempt to fix the reply queue error. -LEJ1 had been failing since 14th of June. My guess is due to some maintenance related issue. Both system tests are running smoothly again after cleaning up their artifacts. - -Meanwhile, maintenance on VPS1-LEJ1 is still on-going and causing headaches. Another password refresh of the access node caused a migration to fail during post-cleanup and I had to set the host and node of the instance to the correct node directly via MySQL. I should apply Rutgers' suggestion of saving some variables in the OS_\* env vars, rather sooner than later. - -Tested retention backups being scheduled, and deployed it to prod. - -Investigate whether we can listen for notifications by Nova so that we can listen for server deletions and remove them from periodic backups. There should be no need to delete backups for the server, as the retention policy will have them deleted after a while. \ No newline at end of file diff --git a/0 inbox/2024-06-26.md b/0 inbox/2024-06-26.md deleted file mode 100644 index e69de29..0000000 diff --git a/0 inbox/2024-08-21.md b/0 inbox/2024-08-21.md deleted file mode 100644 index 7408220..0000000 --- a/0 inbox/2024-08-21.md +++ /dev/null @@ -1,27 +0,0 @@ -Need to start learning to take notes consistently. My idea now, to kick-off this process, is to open a daily note in obsidian every day at the start of the day in which I will keep crude, quick notes. -Then at the end of the day I shall take some time to go over the notes and refine them. Not yet sure if that refinement will already be the end result or if i should go over refined notes again at maybe the end of each week to see if they really stick. -I should commit notes to git to save them. - -Use taskwarrior to keep track of todo's. -`task [ | ]` -commands can be abbreviated as long as they dont become ambigious. -Tasks can be recurring. -See `man task`, specially the CONTEXT section, because i'm always confused about the context, `task context none` to unset the damn thing. -Annotate tasks that are critical with the critical tag, `task annotate +critical` or `task add +critical blaat` -Upgraded to taskwarrior3. Should set-up a task server to synchronize tasks to so I don't lose all of them if my laptop dies. - -Don't forget to always read man pages. Maybe I should create a detailed note about how to use man pages. - -I will start going to the office full-time from now on. -I should make sure my Obsidian notes and taskwarrior tasks are kept safely in a git repo in case i lose my laptop.l - -Today I learned that we can influence the output of `rabbitmqctl list_queues` , we can append it with keywords of columns that it should show, this also allows us to reorder them to set those pesky names on the end to have nicer formatting. -`rabbitmqctl list_queues messages consumers name` shows messages accumulated in Q and the number of consumers it has and then ends with the name. Of course this is explained in the manpage which I should read more often as noted before. - -I should ask for a diagram of the current systems and take note of it, e.g what is OneCRM / OneHOP / OneHome and all that. - -To upgrade a single package using apt we can use `apt-get install --only-upgrade `. - -Need to "relearn" to design stuff and document it. - -Designing structures and a mechanism to return and store info about individual disk backups, we can send that as parent when making incremental backups. Currently we fetch parent reference from checkpoint, which in case of volume's doesn't work of course. It also breaks if, on the testpod, we reset all backups etc but dont do so with all checkpoints. \ No newline at end of file diff --git a/0 inbox/2024-08-22.md b/0 inbox/2024-08-22.md deleted file mode 100644 index fd1531c..0000000 --- a/0 inbox/2024-08-22.md +++ /dev/null @@ -1,25 +0,0 @@ -Woke up full of motivation to go to the office. Asked my honey if it's okay to join her by train "is there enough time left?", "yeah" she replied. Then when I come downstairs and she asks me if I can be fast, I tell her maybe it's better you go then bc i worry I will not be able to. Right away she storms off mad and angry... My motivation gone, so I'll work from home.. - -Went to the office anyway by motorcycle, cannot feel like I'm waking up when staying at home anymore. - -**MermaidJS ZenUML** -Optional **Title** -**Participants** can be declared when used or at the start. -**Annotators** give **participants** a special icon, defined at the doc site. -**Aliases** give a short **participant** name a more descriptive name when rendered, e.g: A as Alice <- Alice will be rendered, but the code can use A. -**Messages** between **participants** can be **sync, async, creation or reply**. -**Sync** use A.\ have a block {} for nested messages -**Async** use an arrow A->B: How are you? -**Creation** new keyword new A(optional params) -**Reply** three ways to define: -* Assign var from **Sync** message, e.g: OptionalType a = A.SyncMessage() -* Return A.SyncMessage { return blaat } -* @return or @reply on an **async** message -**Nesting** can be done on both creation and sync messages using blocks {}. -**Comments** can be added using `// comment` and support markdown. -**Loops** can be defined using keywords: **while, for, foreach and loop**. -**Alternative** paths can be sequenced using **if, else** statements. if(blaat) { A.syncMessage(B) } -**Opt** fragments rendered via `opt { ... }`. This can be useful for optional responses or paths. -**Parallel** paths can be defined using `par { ... }`. -**Try/catch/finally** statements are supported as well. -**@Starter(participant)** to make the line start from a **participant** \ No newline at end of file diff --git a/0 inbox/2024-08-23.md b/0 inbox/2024-08-23.md deleted file mode 100644 index 6f41ca2..0000000 --- a/0 inbox/2024-08-23.md +++ /dev/null @@ -1,6 +0,0 @@ -Discussed about Automatic Evac (HA). We were unsure about what to do when a local storage goes down. Do we restore the instances from backups to a new node? In that case a DB server might lose important transtactions. So we could also wait until it is fixed. - -Would like Rutger to explain the Freenet BGP load balancing setup in more detail. How do the routers work, what are all those interfaces attached to them? - -A local storage disk backup is stored as a QCOW2 image. An incremental has a relationship to its parent, because they neged to be rebased onto each other. -A volume disk backup is stored as raw data and is accompanied by a file of checksums per block. The extra file is used both to track dirtied blocks for incremental backup. \ No newline at end of file diff --git a/0 inbox/2024-08-26.md b/0 inbox/2024-08-26.md deleted file mode 100644 index a91a1c9..0000000 --- a/0 inbox/2024-08-26.md +++ /dev/null @@ -1 +0,0 @@ -Nothing of note \ No newline at end of file diff --git a/0 inbox/2024-08-27.md b/0 inbox/2024-08-27.md deleted file mode 100644 index eb75d3f..0000000 --- a/0 inbox/2024-08-27.md +++ /dev/null @@ -1,11 +0,0 @@ -Changed a few things in the backup service: -* No longer delete all checkpoints when we make a full backup - * TODO: make sure we delete the checkpoint corresponding to a backup when the backup is deleted. - * TODO: Gracefully handle all checkpoints being deleted by openstack -* No longer explicitly specify "incremental" bool, just make it a best effort attempt. - * Still need to work out restores - - -When can we reboot n01.compute vps1-cph8. Its uptime is already 300 days. - -Short call with Jakub. Their priorities have shifted again, the focus on backup will be moved to improving performance of OneHome. They plan to synchronize data from OpenStack to OneHome so they can serve it faster. He asked me whether the customer can influence OpenStack state from within their VM. I told him that probably the only thing they can do to influence OpenStack is shutdown their VM. \ No newline at end of file diff --git a/0 inbox/2024-08-28.md b/0 inbox/2024-08-28.md deleted file mode 100644 index ac0cac1..0000000 --- a/0 inbox/2024-08-28.md +++ /dev/null @@ -1 +0,0 @@ -Had a meeting about onehome performance, they query everything in existence for a customer because of their graphQL workflow. they want to now sync everything in existence to a table so their gQL is fast again. \ No newline at end of file diff --git a/0 inbox/2024-08-29.md b/0 inbox/2024-08-29.md deleted file mode 100644 index e69de29..0000000 diff --git a/0 inbox/2024-09-03.md b/0 inbox/2024-09-03.md deleted file mode 100644 index 1d8a68e..0000000 --- a/0 inbox/2024-09-03.md +++ /dev/null @@ -1 +0,0 @@ -one on one Rutger; diff --git a/0 inbox/2024-09-10.md b/0 inbox/2024-09-10.md deleted file mode 100644 index 07ff5ba..0000000 --- a/0 inbox/2024-09-10.md +++ /dev/null @@ -1 +0,0 @@ -EYQBGNF95VORXXVVJ6NKFZF expressVPN \ No newline at end of file diff --git a/0 inbox/2024-09-30.md b/0 inbox/2024-09-30.md deleted file mode 100644 index fef578b..0000000 --- a/0 inbox/2024-09-30.md +++ /dev/null @@ -1 +0,0 @@ -- Check freenetvps gobs \ No newline at end of file diff --git a/0 inbox/2024-10-02.md b/0 inbox/2024-10-02.md deleted file mode 100644 index 64c611f..0000000 --- a/0 inbox/2024-10-02.md +++ /dev/null @@ -1,6 +0,0 @@ -* Fixed n06 testpod, due to installing stuff via pip we had some conflicting packages being used. Pip has been purged and stuff seems to work normally now. -* Investigated DNS issues on testpod lb with Erik. -* Fixed some small issues in the backup service - * Deleted servers couldn't be removed from the periodic backup, that is now fixed. - * Less noisy logging by moving some stuff to debug level. -* Deployed image_id change to prod \ No newline at end of file diff --git a/0 inbox/2024-10-04.md b/0 inbox/2024-10-04.md deleted file mode 100644 index 9c77b37..0000000 --- a/0 inbox/2024-10-04.md +++ /dev/null @@ -1,26 +0,0 @@ -What is needed to recreate a deleted server: -* name -* description -* block device mapping -* image properties - * while restoring we need to somehow schedule according to image properties, e.g os_type, or is this already resolved by selecting the correct AZ? - * volumes need to clone volume_image_properties -* port mapping -* availability zone -* flavor -* propertieskjk - - - -port -* name -* description -* allowed_address_pairs -* dns_domain -* dns_name -* extra_dhcp_opts -* fixed_ips -* is_port_security_enabled -* security_group_ids -* qos_policy_id -* network_id \ No newline at end of file diff --git a/0 inbox/2024-10-15.md b/0 inbox/2024-10-15.md deleted file mode 100644 index 8887a01..0000000 --- a/0 inbox/2024-10-15.md +++ /dev/null @@ -1,3 +0,0 @@ -Reminder: -* write SOP for fix_allocation -* write SOP for fixing duplicate connector \ No newline at end of file diff --git a/0 inbox/2024-11-11.md b/0 inbox/2024-11-11.md deleted file mode 100644 index 5f3eedb..0000000 --- a/0 inbox/2024-11-11.md +++ /dev/null @@ -1,3 +0,0 @@ -Sqlalchemy/mysql connections lijken een timeout te hitten -zie: -`https://kibana.one.com/s/openstack/app/discover#/?_g=(filters:!(),refreshInterval:(pause:!t,value:60000),time:(from:now-7d%2Fd,to:now))&_a=(columns:!(),dataSource:(dataViewId:d80a7401-b1fd-43b2-bdfc-79dffb0e8b15,type:dataView),filters:!(),hideChart:!f,interval:auto,query:(language:kuery,query:%22cb3204fa-208f-4b90-95a0-846a9e13852b%22),sort:!(!('@timestamp',desc)))` diff --git a/0 inbox/2024-11-12.md b/0 inbox/2024-11-12.md deleted file mode 100644 index e48d53f..0000000 --- a/0 inbox/2024-11-12.md +++ /dev/null @@ -1,3 +0,0 @@ -Prometheus: -- scrape config: `roles/prometheus/onecom-prometheus-scrape-config-*.json` -- \ No newline at end of file diff --git a/0 inbox/Blkio errors.md b/0 inbox/Blkio errors.md deleted file mode 100644 index 2b666a3..0000000 --- a/0 inbox/Blkio errors.md +++ /dev/null @@ -1,14 +0,0 @@ -Dmesg is spewing blkio errors - -Find out which device: e.g dm-4, then: -``` -sudo virsh list --all --name | while read dom; do sudo virsh dumpxml $dom | grep dm-4 && echo $dom; done -``` -To find which domain -After which: -``` -sudo virsh dumpxml $domain -``` -to find the hostname - -reboot it hard. \ No newline at end of file diff --git a/0 inbox/ELF (files).md b/0 inbox/ELF (files).md deleted file mode 100644 index e69de29..0000000 diff --git a/0 inbox/Group.ONE Backup Service.md b/0 inbox/Group.ONE Backup Service.md deleted file mode 100644 index ebb2c92..0000000 --- a/0 inbox/Group.ONE Backup Service.md +++ /dev/null @@ -1,6 +0,0 @@ -# Ubiquitous language -**Project** The customer or tenant that owns the servers, backups and periodic backups. -**Server** Is the subject of a backup or periodic backup. It has one or more disks assigned. -**Disk** Can either be a local storage disk or shared storage volume. It is attached to a server and can be referenced by its name. -**Backup** Is a snapshot in time of a server. It can be used to restore a server to how it was exactly at that moment in time. -**PeriodicBackup** Is a group of servers that will be backed up periodically. \ No newline at end of file diff --git a/0 inbox/Issues met TC oplossen.md b/0 inbox/Issues met TC oplossen.md deleted file mode 100644 index 4ad3bf8..0000000 --- a/0 inbox/Issues met TC oplossen.md +++ /dev/null @@ -1,25 +0,0 @@ -Probleem: referenties in ovs naar devices die niet meer bestaan. Hij probeert vervolgens een nieuw device aan te maken met een qdisc op zo'n niet bestaande device. - -``` -sudo grep -r 'could not open network device' /var/log/openvswitch/ | grep -oi 'tap[a-z0-9\-]*' | uniq > ports_no_device -``` -``` -sudo ip l l | grep ports_no_device | grep -oi 'tap[a-z0-9\-]*' > ports_with_device -``` -``` -for P in $(cat ports_no_device | grep -v -f ports_with_device); do sudo ovs-vsctl del-port $P; done -``` -``` -sudo systemctl restart ovn-controller -``` - - -Rutger versie: -``` -sudo grep 'could not open network device' /var/log/openvswitch/ovs-vswitchd.log | grep -oi 'tap[a-z0-9\-]*' | sort | uniq | while read dev; do ip l l $dev || sudo ovs-vsctl del-port $dev; done; sudo systemctl restart ovn-controller -``` - - -Dus: ovs-vsctl del-port is nice. - -Niet vergeten ovn-controller te herstarten. diff --git a/0 inbox/Neutron DNS & Designate.md b/0 inbox/Neutron DNS & Designate.md deleted file mode 100644 index 48785ab..0000000 --- a/0 inbox/Neutron DNS & Designate.md +++ /dev/null @@ -1,3 +0,0 @@ -https://docs.openstack.org/neutron/latest/admin/config-dns-int.html#config-dns-int - -Lijkt er op dat dns_domain overruled kan worden op port niveau. \ No newline at end of file diff --git a/0 inbox/Nix Flakes.md b/0 inbox/Nix Flakes.md deleted file mode 100644 index cb7f69b..0000000 --- a/0 inbox/Nix Flakes.md +++ /dev/null @@ -1,21 +0,0 @@ -#nix - -Nix flakes are source trees containing a file `flake.nix` at their root. The file `flake.nix` provides a standardized way to provide [[Zettelkast/Index/Nix]] artifacts. It's like a package manager for [[Zettelkast/Index/Nix]]. A flake can be dependent on other Flakes and it's possible to pin dependencies to exact revisions by using a `flake.lock` file. - -Nix flake evaluation is hermetic, meaning that it produces the same result wherever it's built.k - -The feature can be enabled in `~/.config/nix/nix.conf`: - -``` -experimental-features = nix-command flakes -``` - -To initialize in a repo: `nix flake init`. - -In flakes dependencies have to be specified explicitly and MUST be locked to specific versions therefore it's no longer allowed to use the nixpkgs found in `NIX_PATH` by referencing it like ``. - -Output of a Flake is an arbitrary [[Zettelkast/Index/Nix]] value such as a package, [[NixOS]] module or library function. -Commands `nix build` and `nix shell` will build the output `packages..default` unless we specify another output, for example: `nix shell .#checks.aarch64-linux.build`. - ---- -[Flakes Wiki](https://nixos.wiki/wiki/Flakes) \ No newline at end of file diff --git a/0 inbox/Nixpkgs Overlays.md b/0 inbox/Nixpkgs Overlays.md deleted file mode 100644 index e69de29..0000000 diff --git a/0 inbox/Nova Scheduler.md b/0 inbox/Nova Scheduler.md deleted file mode 100644 index 1f58400..0000000 --- a/0 inbox/Nova Scheduler.md +++ /dev/null @@ -1,2 +0,0 @@ -Applies Filters to filters compute hosts that are not usable until a list remains of usable compute hosts. -Filters have different costs, thus it's a good idea to put lightweight filters first as to reduce the load on the heavyweight ones. \ No newline at end of file diff --git a/0 inbox/Ruby gems nixpkgs.md b/0 inbox/Ruby gems nixpkgs.md deleted file mode 100644 index e69de29..0000000 diff --git a/0 inbox/Shell.nix python.md b/0 inbox/Shell.nix python.md deleted file mode 100644 index 7dabd4d..0000000 --- a/0 inbox/Shell.nix python.md +++ /dev/null @@ -1,15 +0,0 @@ -https://nixos.org/manual/nixpkgs/stable/#python - -```nix -{ pkgs ? import {}}: - -pkgs.mkShell { - packages = [ pkgs.virtualenv ]; -} -``` - -```nix -with import {}; -( -let my_toolz = python311.pkgs.buildPythonPackage rec { pname = "toolz"; version = "0.10.0"; pyproject = true; src = fetchPypi { inherit pname version; hash = "sha256-CP3V73yWSArRHBLUct4hrNMjWZlvaaUlkpm1QP66RWA="; }; nativeBuildInputs = [ python311.pkgs.setuptools python311.pkgs.wheel ]; # has no tests doCheck = false; meta = { homepage = "https://github.com/pytoolz/toolz/"; description = "List processing tools and functional utilities"; # [...] }; }; in python311.withPackages (ps: with ps; [ numpy my_toolz ]) ).env -``` \ No newline at end of file diff --git a/0 inbox/Timings VPS.md b/0 inbox/Timings VPS.md deleted file mode 100644 index 51857df..0000000 --- a/0 inbox/Timings VPS.md +++ /dev/null @@ -1,4 +0,0 @@ -10 VPS: avg ms over 100 reqs: 225 ms -20 VPS: first request: 247 ms. avg ms over 100 reqs: 203 ms -50 VPS: first request: 259 ms. avg ms over 100 reqs: 220 ms -101 VPS: first request: 264 ms. avg ms over 100 reqs: 221 ms diff --git a/0 inbox/Updates ansible ad-hoc.md b/0 inbox/Updates ansible ad-hoc.md deleted file mode 100644 index 2b8514c..0000000 --- a/0 inbox/Updates ansible ad-hoc.md +++ /dev/null @@ -1,19 +0,0 @@ -update apt cache: -``` -ansible -m ansible.builtin.apt -a "update_cache=true cache_valid_time=3600" -``` - -update packages: -``` -ansible -m ansible.builtin.apt -a "name=* state=latest" -``` - -run puppet: -``` -ansible -a "/opt/puppetlabs/bin/puppet agent --test" -``` - -reboot: -``` -ansible -m ansible.builtin.reboot -f 1 -``` diff --git a/0 inbox/Zoner Migration.md b/0 inbox/Zoner Migration.md deleted file mode 100644 index dc61370..0000000 --- a/0 inbox/Zoner Migration.md +++ /dev/null @@ -1 +0,0 @@ -* Kernel installation \ No newline at end of file diff --git a/0 inbox/antagonist_migratie.md b/0 inbox/antagonist_migratie.md deleted file mode 100644 index c04a00b..0000000 --- a/0 inbox/antagonist_migratie.md +++ /dev/null @@ -1,27 +0,0 @@ -Migrate VMs - -Not managed by customers, only Antagonist managed -Most of it is backoffice, only few are front facing - -Ovirt (KVM) - - -Possible to change IPs? - -Difficulty: private networking between hosts - databases only ones in the private network - - Should be possible to switch IPs - - -Outsourced routers to DC provider, more difficult to integrate networks - -Most of the things can be redeployed, Rutger notes that maybe all they need is access - -We can use their left-over IP space - -Features -- requires internal network, so want to create networks/routers -- HA/automatic failover? -> Not yet -- Anti-affinity rules? Dont want all the LBs on the same (failing) node - diff --git a/0 inbox/intro.md b/0 inbox/intro.md deleted file mode 100644 index 0849f2f..0000000 --- a/0 inbox/intro.md +++ /dev/null @@ -1,21 +0,0 @@ -# Foundational document -This document outlines the basis of my note taking system and serves mostly as a reminder to myself. - -I want to train the skill of effective note taking as I believe that it will serve as a foundation -to any other study and will boost my overall productivity. - -## Why take notes? -By writing down notes we can relate information about a subject in a very personal way using our -own experiences to relate them to. - -We can us our notes as inspiration to create. In a centralized note taking system it is possible -to easily revisit notes and use them to inspire new ideas. - -Another effect of a centralized note system is that it is very easy to search through our notes -and use them as reference material. - - -## Quotes -"If I can't write about something; I don't understand the topic." - Mischa van den Burg? -"Clear thinking becomes clear writing; one can't exist without the other." - William Zinsser. - diff --git a/0 inbox/para.md b/0 inbox/para.md deleted file mode 100644 index 345f597..0000000 --- a/0 inbox/para.md +++ /dev/null @@ -1,10 +0,0 @@ -# PARA method -A method of note taking by Tiago Forte (Building a second brain) - -Projects: Anything that requires the completion of multiple tasks to reach a certain outcome. -Areas: Something that you are responsible for for a longer period of your live. -Resources: Notes about resources that you use to do anything, such as tools and programming languages for example. -Archive: Notes that you feel are nog longer relevant to you. They might end up being useful in unexpected ways -so don't throw away! - -GTD: Getting Things Done diff --git a/3 resources/Group.ONE Infrastructure.md b/3 resources/Group.ONE Infrastructure.md deleted file mode 100644 index 1d9ba33..0000000 --- a/3 resources/Group.ONE Infrastructure.md +++ /dev/null @@ -1 +0,0 @@ -I want to have an overview of systems and what they are responsible for. Example, what is OneHOP or OneHome and what do they do. I plan to write that down in this document. \ No newline at end of file diff --git a/3 resources/Man pages.md b/3 resources/Man pages.md deleted file mode 100644 index 901be15..0000000 --- a/3 resources/Man pages.md +++ /dev/null @@ -1,3 +0,0 @@ -**2024-08-21 18:07:20** -I need to more consistently read man pages. Example is `man task` to see how taskwarrior works, after reading it today I found that it is actually quite easy. -They often contain good information on how to use a certain tool. \ No newline at end of file diff --git a/3 resources/Note taking.md b/3 resources/Note taking.md deleted file mode 100644 index 2a47666..0000000 --- a/3 resources/Note taking.md +++ /dev/null @@ -1,7 +0,0 @@ -**2024-08-21 17:59:00** -I want to become better at taking notes consistently in the hope that I forget less important things and become better overall at managing knowledge. -To try and create the habit I will open a new daily note in Obsidian at the start of the day and sit down at the end of the day to refine anything written down into a more permanent form like this one. I'm hoping that by doing this consistently I will more often write down ideas, information, etc. during the day, rather than forget about it. - -After refining the daily notes, I should commit and push changes to a remote git repo that is also regularly backed up so I keep all this knowledge safe. - -I'm not yet sure if and how I incorporate this into a "zettelkast", something I did start before but never really stuck. I will need to do more research into this, and note taking overall. \ No newline at end of file diff --git a/3 resources/RabbitMQ.md b/3 resources/RabbitMQ.md deleted file mode 100644 index e9c94c5..0000000 --- a/3 resources/RabbitMQ.md +++ /dev/null @@ -1,2 +0,0 @@ -**2024-08-21 18:09** -With `rabbitmqctl list_queues` we can add columns to show. So instead of the default layout which shows messages in the Q after the name and is horrible we can specify stuff like: `rabbitmqctl list_queues messages consumers name`. There are more options to be found in `man rabbitmqctl`. diff --git a/3 resources/TaskWarrior (ToDo).md b/3 resources/TaskWarrior (ToDo).md deleted file mode 100644 index 6002fde..0000000 --- a/3 resources/TaskWarrior (ToDo).md +++ /dev/null @@ -1,3 +0,0 @@ -**2024-08-21 18:03:00** -I want to better keep track of what i'm doing, for that purpose I will use taskwarrior. For usage see `man task`. -Adding a task is done by `task add`, to modify it `task modify` . I use priorities, they can be set on a new or existing task with `priority:H/M/L`. diff --git a/3 resources/ansible.md b/3 resources/ansible.md new file mode 100644 index 0000000..501f9d0 --- /dev/null +++ b/3 resources/ansible.md @@ -0,0 +1,20 @@ +# Ad-hoc commands +update apt cache: +``` +ansible -m ansible.builtin.apt -a "update_cache=true cache_valid_time=3600" +``` + +update packages: +``` +ansible -m ansible.builtin.apt -a "name=* state=latest" +``` + +run puppet: +``` +ansible -a "/opt/puppetlabs/bin/puppet agent --test" +``` + +reboot: +``` +ansible -m ansible.builtin.reboot -f 1 +``` diff --git a/3 resources/elixir.md b/3 resources/elixir.md new file mode 100644 index 0000000..4d38de1 --- /dev/null +++ b/3 resources/elixir.md @@ -0,0 +1,228 @@ +Elixir supports "macro's", which is Elixir code that runs at compile time. They receive the AST of the source code as input and can apply transformations to it. This is how Elixir is written itself, using Elixir macro's. + +Elixir functions can be organized into [[elixir - modules]]. +``` +defmodule MyModule do + # Comment + @moduledoc "Documentation for the module" + import IO + alias IO, as MyIO # Alias import + + @pi 3.14 # Module attribute + + @doc "Describe what the function does" + def hello do + MyIO.puts("blaat") + 3*@pi # Module attribute reference + done + + @spec hello_private(string) # typespec, can be used by dialyzer, very useful + defp hello_private(hello) do + puts(hello) + done +done +``` + +The [[Elixir - Kernel]] module is always imported, so functions we use without prefix come from Kernel. + +Elixir introduces a concept called [[Elixir - Atoms]]. They are named constants, like enumerations in C. +``` +:blaat +:"Bla at" +Blaat == :"Elixir.Blaat" # Alias + +var = :blaat # var contains only a reference to the atom, thus is small and fast. +``` + +[[Elixir - Aliases]] are internally represented as "Elixir.RealThing", in the case of the module alias above `MyIO == Elixir.IO`. + +[[Elixir - Tuples]] group a, usually small, fixed number of elements together. Kernel.elem/2 for access, Kernel.put_elem/3 for updating. + +[[Elixir - Lists]] are a recursive structure with a head of any type and a tail which is another list. They are notated as regular arrays in other languages, but can be written as `[head | tail]`. Therefore it's easy and efficient to push a new item to the top, we can use the head | tail notation -> +https://hexdocs.pm/elixir/List.html +https://hexdocs.pm/elixir/Enum.html +``` +list = [1, 2, 3] +list = [4 | list] +list +[4, 1, 2, 3] +``` + +[[Elixir - Immutable functions]] give a form of atomicity, because operations (other functions) they call do not mutate data if anything fails we can just return the original data without having changed anything. + +[[Elixir - Maps]] %{}, can also be created with Map.new([{1, 2}, {3, 4}]) (2-tuples). +https://hexdocs.pm/elixir/Map.html +``` +a = %{1 => 1} +a[1] +1 +``` +Maps can also be used to define structures, +``` +person = %{name: "Jasper"} +``` + +[[Elixir - Binaries]] are consecutive sequences of bytes. +``` +<<1, 1>> is a two-byte binary where each byte has value of, thus +0000 0001 0000 0001 +<<1>> <> <<2>> concatenate two binaries +0000 0001 0000 0010 +``` + +[[Elixir - Strings]] +``` +"This is a stringt" +" +Multiline +string +" +a = 1337 +"Print number #{a}" # #{} allows evaluation of values strings + +~s(This is a sigil which is also a string) +~s(Is useful to "use" quotes) +str = "INTERPOLATION!" +~S(Capital-S sigil allows prevention of string #{str} and \nescaping) +"Capital-S sigil allows prevention of string \#{str} and \\nescaping" + +"Blaat" <> " henk" # Concatenation works like binaries, because strings are binaries +``` + +[[Elixir - Lambda]] +``` +square = fn x -> # lambda's use fn + x*x +end + +square.(5) # lambda is called with name period arguments enclosed by parens. The dot is to make it known that we are calling a lambda and not a regular function. +``` + +For cases where the lambda just forwards its arguments to another function there is a special syntax, example: +[[Elixir - capture operator]] +``` +Enum.each([1, 2, 3], fn x -> IO.puts(x) end) +Enum.each([1, 2, 3], &IO.puts/1) # The & is called the "capture operator" and can also be used to shorten a lambda definition: +lambda = fn x, y, z -> x * y + z end +lambda = &(&1 * &2 + &3) # Like bash arguments ${1} ${2} etc +``` + +[[Elixir - Closure]] +A lambda can reference variables from the outside scope. If we rebind the variable in the parent scope, the lambda will still reference the old one. +``` +outside = "Abc" +lambda = &IO.puts/1 +outside = "cdef" +lambda.() +"Abc" +``` + +Range +0..1, internally represented as a map with bounds set, therefore small no matter how "big" the range. Is also an enumeration so can use the Enum module. + +Keyword list +List of 2-tuple where the 1st element is an atom. E.g. `[{:monday, 1}, {:tuesday, 2}]` can be written more elegant as `[monday: 1, tuesday: 2]` +https://hexdocs.pm/elixir/Keyword.html +Can be used as kwargs like in python. + +MapSet +https://hexdocs.pm/elixir/MapSet.html +Also an enumeration. +Initialize with MapSet.new + +Times and Dates +Have modules: Date, Time, DateTime, NaiveDateTime +Created with sigil ~D for dates, and ~T for time +``` +dt = ~D[2023-01-01] +dt.year +2023 + +tm = ~T[19:03:32] +tm.second +32 +``` + +IO lists +Are lists that can consist of one of three types: +- Int in range 0..255 +- Binaries +- Another IO list +It is thus a tree. Input operations are O(1) + + +Pattern matching +The '=' operator is not an assignment operator, but a match operator. +pattern = expression +Pattern can be list, map, tuple, variable, binaries, binary strings +constants, atom can be matched to discriminate results of expressions. +{:ok, result} = expr, fails is expr returns for example {:error, result} +Patterns can be nested: `{_, {hour, _, }, _} = :calendar.local_time()` + +Maps can partial match, to extract a property from a complex map. +Lists can abuse their recursive naturs: `[head | tail] = [1, 2, 3] head = 1, tail = [2,3]` +Pin-operator `^` is used to match against the value of a variable: +``` +a = "Bob" +{^a, _} = {"Bob", 25} <- Matches because the value of a is "Bob" +{^a, _} = {"Alice", 25} <- Doesn't match +``` + +Pattern matching using strings it's possible to match the beginning of a string and assign the rest to a var: +``` +command = "ping www.hostnet.nl" +"ping " <> url = command +url = "www.hostnet.nl" +``` + +Pattern matching can be done in function arguments and enabled "multiclause functions", which is a sort of function overloading. It's multiple definitions of the same function, with the same arity, but with different argument patterns. They are treated as a single function, so with the capture operator you can use all "variants". +``` +defmodule Geo do + def area({:square, a, b}), do a * b end + def area({:circle, r}), do r * r * pi end + def area(unknown), do {:error, {:unknown_shape, unknown}} end + # ^ do mind that the arity has to match for this catch-all error, also + # ordering is important. The runtime matches from top to bottom. +end + +fn = &Geo.area/1 + +fn.({:square, 1, 2}) +2 +fn.({:circle, 23}) +whatever this is +``` + +Conditionals can be implemented using multiclause functions, but also with the regular if..else statements, cond do .. end and case expression do ... end. + +A with-clause can be used to match multiple patterns in order and halt if a pattern doesn't match. +``` +defp extract_login(%{"login" => login}%) do, %{:ok, login} end +defp extract_login(_) do, {:error, "login missing"} end + +defp extract_email(%{"email" => email}) do, %{:ok, email} end +defp extract_email(_) do, %{:error, "email missing"} end + +def extract_user(user) do + case extract_login(user) do + {:error, reason} -> {:error, reason} + {:ok, login} -> + case extract_email(user) do + {:error, reason} -> {:error, reason} + {ok, email} -> %{login: login, email: email} + end + end +end + +# can be written as + +def extract_user(user) do + with {:ok, login} <- extract_login(user), + {:ok, email} <- extract_email(user) do + {:ok, %{login: login, email: email}} + end +end +``` + +Looping is mainly implemented via recursion. The break condition is implemented via a multiclause function matching the condition that you want to break at. +Recursion can be expensive, unless the recursive call is at the end of a function, which is called a tail-call. Tail-calls are optimized to not require any additional memory, because their result is also the result of the caller, so we don't need to come back to the caller. \ No newline at end of file diff --git a/3 resources/group.one infra.md b/3 resources/group.one infra.md new file mode 100644 index 0000000..1d9ba33 --- /dev/null +++ b/3 resources/group.one infra.md @@ -0,0 +1 @@ +I want to have an overview of systems and what they are responsible for. Example, what is OneHOP or OneHome and what do they do. I plan to write that down in this document. \ No newline at end of file diff --git a/3 resources/man pages.md b/3 resources/man pages.md new file mode 100644 index 0000000..576d4b8 --- /dev/null +++ b/3 resources/man pages.md @@ -0,0 +1,2 @@ +I need to more consistently read man pages. Example is `man task` to see how taskwarrior works, after reading it today I found that it is actually quite easy. +They often contain good information on how to use a certain tool. \ No newline at end of file diff --git a/3 resources/mermaidjs.md b/3 resources/mermaidjs.md new file mode 100644 index 0000000..e468459 --- /dev/null +++ b/3 resources/mermaidjs.md @@ -0,0 +1,21 @@ +**ZenUML** +Optional **Title** +**Participants** can be declared when used or at the start. +**Annotators** give **participants** a special icon, defined at the doc site. +**Aliases** give a short **participant** name a more descriptive name when rendered, e.g: A as Alice <- Alice will be rendered, but the code can use A. +**Messages** between **participants** can be **sync, async, creation or reply**. +**Sync** use A.\ have a block {} for nested messages +**Async** use an arrow A->B: How are you? +**Creation** new keyword new A(optional params) +**Reply** three ways to define: +* Assign var from **Sync** message, e.g: OptionalType a = A.SyncMessage() +* Return A.SyncMessage { return blaat } +* @return or @reply on an **async** message +**Nesting** can be done on both creation and sync messages using blocks {}. +**Comments** can be added using `// comment` and support markdown. +**Loops** can be defined using keywords: **while, for, foreach and loop**. +**Alternative** paths can be sequenced using **if, else** statements. if(blaat) { A.syncMessage(B) } +**Opt** fragments rendered via `opt { ... }`. This can be useful for optional responses or paths. +**Parallel** paths can be defined using `par { ... }`. +**Try/catch/finally** statements are supported as well. +**@Starter(participant)** to make the line start from a **participant** \ No newline at end of file diff --git a/3 resources/nix.md b/3 resources/nix.md new file mode 100644 index 0000000..d407d2e --- /dev/null +++ b/3 resources/nix.md @@ -0,0 +1,58 @@ +# Language + `nix repl` to interactively evaluate Nix expressions. `:p` if output is not full. + `nix-instantiate --eval .nix` to evaluate a Nix expression from a file. `--strict`. + +Nix is like JSON, but with functions. +Recursive attribute sets can reference values declared earlier in the same set. +```nix +rec { + one = 1; + two = one + 1; +} +``` + +A `let` binding is used to assign names to values just as attribute sets, they can then be used in expressions. Let bindings have a local scope. + +A `with` allows referencing attributes of attribute sets without referencing the set. +# Flakes +Nix flakes are source trees containing a file `flake.nix` at their root. The file `flake.nix` provides a standardized way to provide [[Zettelkast/Index/Nix]] artifacts. It's like a package manager for [[Zettelkast/Index/Nix]]. A flake can be dependent on other Flakes and it's possible to pin dependencies to exact revisions by using a `flake.lock` file. + +Nix flake evaluation is hermetic, meaning that it produces the same result wherever it's built.k + +The feature can be enabled in `~/.config/nix/nix.conf`: + +``` +experimental-features = nix-command flakes +``` + +To initialize in a repo: `nix flake init`. + +In flakes dependencies have to be specified explicitly and MUST be locked to specific versions therefore it's no longer allowed to use the nixpkgs found in `NIX_PATH` by referencing it like ``. + +Output of a Flake is an arbitrary [[Zettelkast/Index/Nix]] value such as a package, [[NixOS]] module or library function. +Commands `nix build` and `nix shell` will build the output `packages..default` unless we specify another output, for example: `nix shell .#checks.aarch64-linux.build`. + +# Overlays +[[TODO]] + +# Shell.nix +Can be used to set up per dir environments, e.g using direnv to automatically activate them. +## Python +https://nixos.org/manual/nixpkgs/stable/#python + +```nix +{ pkgs ? import {}}: + +pkgs.mkShell { + packages = [ pkgs.virtualenv ]; +} +``` + +```nix +with import {}; +( +let my_toolz = python311.pkgs.buildPythonPackage rec { pname = "toolz"; version = "0.10.0"; pyproject = true; src = fetchPypi { inherit pname version; hash = "sha256-CP3V73yWSArRHBLUct4hrNMjWZlvaaUlkpm1QP66RWA="; }; nativeBuildInputs = [ python311.pkgs.setuptools python311.pkgs.wheel ]; # has no tests doCheck = false; meta = { homepage = "https://github.com/pytoolz/toolz/"; description = "List processing tools and functional utilities"; # [...] }; }; in python311.withPackages (ps: with ps; [ numpy my_toolz ]) ).env +``` + +--- +[Flakes Wiki](https://nixos.wiki/wiki/Flakes) \ No newline at end of file diff --git a/3 resources/note taking system.md b/3 resources/note taking system.md new file mode 100644 index 0000000..907e2ce --- /dev/null +++ b/3 resources/note taking system.md @@ -0,0 +1,21 @@ +## Why take notes? +By writing down notes we can relate information about a subject in a very personal way using our own experiences to relate them to. + +We can us our notes as inspiration to create. In a centralized note taking system it is possible to easily revisit notes and use them to inspire new ideas. + +Another effect of a centralized note system is that it is very easy to search through our notes and use them as reference material. + +## How +Using the para method; a method of note taking by Tiago Forte (Building a second brain) + +Projects: Anything that requires the completion of multiple tasks to reach a certain outcome. +Areas: Something that you are responsible for for a longer period of your live. +Resources: Notes about resources that you use to do anything, such as tools and programming languages for example. +Archive: Notes that you feel are nog longer relevant to you. They might end up being useful in unexpected ways +so don't throw away! + +GTD: Getting Things Done + +## Quotes +"If I can't write about something; I don't understand the topic." - Mischa van den Burg? +"Clear thinking becomes clear writing; one can't exist without the other." - William Zinsser. diff --git a/3 resources/programming/Elixir - modules.md b/3 resources/programming/Elixir - modules.md deleted file mode 100644 index e69de29..0000000 diff --git a/3 resources/programming/Elixir.md b/3 resources/programming/Elixir.md deleted file mode 100644 index b106ae4..0000000 --- a/3 resources/programming/Elixir.md +++ /dev/null @@ -1,228 +0,0 @@ -Elixir supports "macro's", which is Elixir code that runs at compile time. They receive the AST of the source code as input and can apply transformations to it. This is how Elixir is written itself, using Elixir macro's. - -Elixir functions can be organized into [[Elixir - modules]]. -``` -defmodule MyModule do - # Comment - @moduledoc "Documentation for the module" - import IO - alias IO, as MyIO # Alias import - - @pi 3.14 # Module attribute - - @doc "Describe what the function does" - def hello do - MyIO.puts("blaat") - 3*@pi # Module attribute reference - done - - @spec hello_private(string) # typespec, can be used by dialyzer, very useful - defp hello_private(hello) do - puts(hello) - done -done -``` - -The [[Elixir - Kernel]] module is always imported, so functions we use without prefix come from Kernel. - -Elixir introduces a concept called [[Elixir - Atoms]]. They are named constants, like enumerations in C. -``` -:blaat -:"Bla at" -Blaat == :"Elixir.Blaat" # Alias - -var = :blaat # var contains only a reference to the atom, thus is small and fast. -``` - -[[Elixir - Aliases]] are internally represented as "Elixir.RealThing", in the case of the module alias above `MyIO == Elixir.IO`. - -[[Elixir - Tuples]] group a, usually small, fixed number of elements together. Kernel.elem/2 for access, Kernel.put_elem/3 for updating. - -[[Elixir - Lists]] are a recursive structure with a head of any type and a tail which is another list. They are notated as regular arrays in other languages, but can be written as `[head | tail]`. Therefore it's easy and efficient to push a new item to the top, we can use the head | tail notation -> -https://hexdocs.pm/elixir/List.html -https://hexdocs.pm/elixir/Enum.html -``` -list = [1, 2, 3] -list = [4 | list] -list -[4, 1, 2, 3] -``` - -[[Elixir - Immutable functions]] give a form of atomicity, because operations (other functions) they call do not mutate data if anything fails we can just return the original data without having changed anything. - -[[Elixir - Maps]] %{}, can also be created with Map.new([{1, 2}, {3, 4}]) (2-tuples). -https://hexdocs.pm/elixir/Map.html -``` -a = %{1 => 1} -a[1] -1 -``` -Maps can also be used to define structures, -``` -person = %{name: "Jasper"} -``` - -[[Elixir - Binaries]] are consecutive sequences of bytes. -``` -<<1, 1>> is a two-byte binary where each byte has value of, thus -0000 0001 0000 0001 -<<1>> <> <<2>> concatenate two binaries -0000 0001 0000 0010 -``` - -[[Elixir - Strings]] -``` -"This is a stringt" -" -Multiline -string -" -a = 1337 -"Print number #{a}" # #{} allows evaluation of values strings - -~s(This is a sigil which is also a string) -~s(Is useful to "use" quotes) -str = "INTERPOLATION!" -~S(Capital-S sigil allows prevention of string #{str} and \nescaping) -"Capital-S sigil allows prevention of string \#{str} and \\nescaping" - -"Blaat" <> " henk" # Concatenation works like binaries, because strings are binaries -``` - -[[Elixir - Lambda]] -``` -square = fn x -> # lambda's use fn - x*x -end - -square.(5) # lambda is called with name period arguments enclosed by parens. The dot is to make it known that we are calling a lambda and not a regular function. -``` - -For cases where the lambda just forwards its arguments to another function there is a special syntax, example: -[[Elixir - capture operator]] -``` -Enum.each([1, 2, 3], fn x -> IO.puts(x) end) -Enum.each([1, 2, 3], &IO.puts/1) # The & is called the "capture operator" and can also be used to shorten a lambda definition: -lambda = fn x, y, z -> x * y + z end -lambda = &(&1 * &2 + &3) # Like bash arguments ${1} ${2} etc -``` - -[[Elixir - Closure]] -A lambda can reference variables from the outside scope. If we rebind the variable in the parent scope, the lambda will still reference the old one. -``` -outside = "Abc" -lambda = &IO.puts/1 -outside = "cdef" -lambda.() -"Abc" -``` - -Range -0..1, internally represented as a map with bounds set, therefore small no matter how "big" the range. Is also an enumeration so can use the Enum module. - -Keyword list -List of 2-tuple where the 1st element is an atom. E.g. `[{:monday, 1}, {:tuesday, 2}]` can be written more elegant as `[monday: 1, tuesday: 2]` -https://hexdocs.pm/elixir/Keyword.html -Can be used as kwargs like in python. - -MapSet -https://hexdocs.pm/elixir/MapSet.html -Also an enumeration. -Initialize with MapSet.new - -Times and Dates -Have modules: Date, Time, DateTime, NaiveDateTime -Created with sigil ~D for dates, and ~T for time -``` -dt = ~D[2023-01-01] -dt.year -2023 - -tm = ~T[19:03:32] -tm.second -32 -``` - -IO lists -Are lists that can consist of one of three types: -- Int in range 0..255 -- Binaries -- Another IO list -It is thus a tree. Input operations are O(1) - - -Pattern matching -The '=' operator is not an assignment operator, but a match operator. -pattern = expression -Pattern can be list, map, tuple, variable, binaries, binary strings -constants, atom can be matched to discriminate results of expressions. -{:ok, result} = expr, fails is expr returns for example {:error, result} -Patterns can be nested: `{_, {hour, _, }, _} = :calendar.local_time()` - -Maps can partial match, to extract a property from a complex map. -Lists can abuse their recursive naturs: `[head | tail] = [1, 2, 3] head = 1, tail = [2,3]` -Pin-operator `^` is used to match against the value of a variable: -``` -a = "Bob" -{^a, _} = {"Bob", 25} <- Matches because the value of a is "Bob" -{^a, _} = {"Alice", 25} <- Doesn't match -``` - -Pattern matching using strings it's possible to match the beginning of a string and assign the rest to a var: -``` -command = "ping www.hostnet.nl" -"ping " <> url = command -url = "www.hostnet.nl" -``` - -Pattern matching can be done in function arguments and enabled "multiclause functions", which is a sort of function overloading. It's multiple definitions of the same function, with the same arity, but with different argument patterns. They are treated as a single function, so with the capture operator you can use all "variants". -``` -defmodule Geo do - def area({:square, a, b}), do a * b end - def area({:circle, r}), do r * r * pi end - def area(unknown), do {:error, {:unknown_shape, unknown}} end - # ^ do mind that the arity has to match for this catch-all error, also - # ordering is important. The runtime matches from top to bottom. -end - -fn = &Geo.area/1 - -fn.({:square, 1, 2}) -2 -fn.({:circle, 23}) -whatever this is -``` - -Conditionals can be implemented using multiclause functions, but also with the regular if..else statements, cond do .. end and case expression do ... end. - -A with-clause can be used to match multiple patterns in order and halt if a pattern doesn't match. -``` -defp extract_login(%{"login" => login}%) do, %{:ok, login} end -defp extract_login(_) do, {:error, "login missing"} end - -defp extract_email(%{"email" => email}) do, %{:ok, email} end -defp extract_email(_) do, %{:error, "email missing"} end - -def extract_user(user) do - case extract_login(user) do - {:error, reason} -> {:error, reason} - {:ok, login} -> - case extract_email(user) do - {:error, reason} -> {:error, reason} - {ok, email} -> %{login: login, email: email} - end - end -end - -# can be written as - -def extract_user(user) do - with {:ok, login} <- extract_login(user), - {:ok, email} <- extract_email(user) do - {:ok, %{login: login, email: email}} - end -end -``` - -Looping is mainly implemented via recursion. The break condition is implemented via a multiclause function matching the condition that you want to break at. -Recursion can be expensive, unless the recursive call is at the end of a function, which is called a tail-call. Tail-calls are optimized to not require any additional memory, because their result is also the result of the caller, so we don't need to come back to the caller. \ No newline at end of file diff --git a/3 resources/programming/Nix.md b/3 resources/programming/Nix.md deleted file mode 100644 index fca31a8..0000000 --- a/3 resources/programming/Nix.md +++ /dev/null @@ -1,16 +0,0 @@ - `nix repl` to interactively evaluate Nix expressions. `:p` if output is not full. - `nix-instantiate --eval .nix` to evaluate a Nix expression from a file. `--strict`. - -Nix is like JSON, but with functions. -Recursive attribute sets can reference values declared earlier in the same set. -```nix -rec { - one = 1; - two = one + 1; -} -``` - -A `let` binding is used to assign names to values just as attribute sets, they can then be used in expressions. Let bindings have a local scope. - -A `with` allows referencing attributes of attribute sets without referencing the set. - diff --git a/3 resources/rabbitmq.md b/3 resources/rabbitmq.md new file mode 100644 index 0000000..312bec2 --- /dev/null +++ b/3 resources/rabbitmq.md @@ -0,0 +1 @@ +With `rabbitmqctl list_queues` we can add columns to show. So instead of the default layout which shows messages in the Q after the name and is horrible we can specify stuff like: `rabbitmqctl list_queues messages consumers name`. There are more options to be found in `man rabbitmqctl`. diff --git a/3 resources/taskwarrior.md b/3 resources/taskwarrior.md new file mode 100644 index 0000000..34fe640 --- /dev/null +++ b/3 resources/taskwarrior.md @@ -0,0 +1,4 @@ +`task add something +label` +`task context none` +`task add highprio thing +label priority:H` +`task context define mycontext +label project:blaat` diff --git a/4 archive/Blkio errors.md b/4 archive/Blkio errors.md new file mode 100644 index 0000000..2b666a3 --- /dev/null +++ b/4 archive/Blkio errors.md @@ -0,0 +1,14 @@ +Dmesg is spewing blkio errors + +Find out which device: e.g dm-4, then: +``` +sudo virsh list --all --name | while read dom; do sudo virsh dumpxml $dom | grep dm-4 && echo $dom; done +``` +To find which domain +After which: +``` +sudo virsh dumpxml $domain +``` +to find the hostname + +reboot it hard. \ No newline at end of file diff --git a/4 archive/Issues met TC oplossen.md b/4 archive/Issues met TC oplossen.md new file mode 100644 index 0000000..4ad3bf8 --- /dev/null +++ b/4 archive/Issues met TC oplossen.md @@ -0,0 +1,25 @@ +Probleem: referenties in ovs naar devices die niet meer bestaan. Hij probeert vervolgens een nieuw device aan te maken met een qdisc op zo'n niet bestaande device. + +``` +sudo grep -r 'could not open network device' /var/log/openvswitch/ | grep -oi 'tap[a-z0-9\-]*' | uniq > ports_no_device +``` +``` +sudo ip l l | grep ports_no_device | grep -oi 'tap[a-z0-9\-]*' > ports_with_device +``` +``` +for P in $(cat ports_no_device | grep -v -f ports_with_device); do sudo ovs-vsctl del-port $P; done +``` +``` +sudo systemctl restart ovn-controller +``` + + +Rutger versie: +``` +sudo grep 'could not open network device' /var/log/openvswitch/ovs-vswitchd.log | grep -oi 'tap[a-z0-9\-]*' | sort | uniq | while read dev; do ip l l $dev || sudo ovs-vsctl del-port $dev; done; sudo systemctl restart ovn-controller +``` + + +Dus: ovs-vsctl del-port is nice. + +Niet vergeten ovn-controller te herstarten. diff --git a/4 archive/antagonist_migratie.md b/4 archive/antagonist_migratie.md new file mode 100644 index 0000000..c04a00b --- /dev/null +++ b/4 archive/antagonist_migratie.md @@ -0,0 +1,27 @@ +Migrate VMs + +Not managed by customers, only Antagonist managed +Most of it is backoffice, only few are front facing + +Ovirt (KVM) + + +Possible to change IPs? + +Difficulty: private networking between hosts + databases only ones in the private network + + Should be possible to switch IPs + + +Outsourced routers to DC provider, more difficult to integrate networks + +Most of the things can be redeployed, Rutger notes that maybe all they need is access + +We can use their left-over IP space + +Features +- requires internal network, so want to create networks/routers +- HA/automatic failover? -> Not yet +- Anti-affinity rules? Dont want all the LBs on the same (failing) node + diff --git a/TODO.md b/TODO.md new file mode 100644 index 0000000..77004b5 --- /dev/null +++ b/TODO.md @@ -0,0 +1 @@ +Used as a link for subjects that I have yet to cover in notes. \ No newline at end of file -- cgit v1.2.3