diff options
author | Jasper Ras <jras@hostnet.nl> | 2025-04-30 15:14:56 +0200 |
---|---|---|
committer | Jasper Ras <jras@hostnet.nl> | 2025-04-30 15:14:56 +0200 |
commit | 4223e9afd4401bd930ca815ffa816eb76fad83f6 (patch) | |
tree | b8907fc631e2ad61b7c8741304b8cd6e6e4a53de | |
parent | f43d0473320b9b2297b1a00880316aee9066b81a (diff) |
vault backup: 2025-04-30 15:14:56
-rw-r--r-- | .trash/OVN Clusters.md | 5 | ||||
-rw-r--r-- | .trash/OVN upgrade Ansible playbook.md | 22 | ||||
-rw-r--r-- | .trash/Untitled.canvas (renamed from Untitled.canvas) | 0 | ||||
-rw-r--r-- | Checklist for network node issues after rebooting.md | 18 | ||||
-rw-r--r-- | Custom OVN debian packaging.md | 7 | ||||
-rw-r--r-- | How to see over which network nodes traffic is being routed.md | 20 | ||||
-rw-r--r-- | OVN Clusters.md | 59 | ||||
-rw-r--r-- | OVN upgrade playbook.md | 29 | ||||
-rw-r--r-- | OVN upgrade.md | 36 | ||||
-rw-r--r-- | Random notes on Freenet deployment.md | 2 | ||||
-rw-r--r-- | Reading a file line-by-line in Python is easy with a for loop.md | 10 | ||||
-rw-r--r-- | Systemd timer.md | 45 | ||||
-rw-r--r-- | Using Ansible wait_for to wait until a file exists.md | 11 |
13 files changed, 175 insertions, 89 deletions
diff --git a/.trash/OVN Clusters.md b/.trash/OVN Clusters.md new file mode 100644 index 0000000..ad3d667 --- /dev/null +++ b/.trash/OVN Clusters.md @@ -0,0 +1,5 @@ +--- +tags: + - work + - groupvps +--- diff --git a/.trash/OVN upgrade Ansible playbook.md b/.trash/OVN upgrade Ansible playbook.md new file mode 100644 index 0000000..7905071 --- /dev/null +++ b/.trash/OVN upgrade Ansible playbook.md @@ -0,0 +1,22 @@ +--- +tags: + - groupvps +--- +- Makes a backup of OVN db's on networking nodes + - Creates files on the local system under /tmp + - Creates files on the remote system under /var/.ovn-backups +- Pins the controller version on both compute and networking nodes + - `ovs-vsctl set open_vswitch . external_ids:ovn-match-northd-version=true` +- Upgrades ovn-central, ovn-common and ovn-host on network nodes + - Installs packages + - restarts ovn-controller, ovn-northd, ovn-ovsdb-server-sb/nb + - Checks ovsdb-server is running + - Checks the socket file for existence +- Upgrades ovn controller on compute nodes + - Installs package + - restart ovn-controller + +**Invocation** +``` +ansible-playbook playbooks/failsafe_install_ovn.yml --limit vps_testpod --extra-vars "ovn_search_version=22.03.8+20250403.1251.35813e0b.systems.jammy1" +```
\ No newline at end of file diff --git a/Untitled.canvas b/.trash/Untitled.canvas index 9e26dfe..9e26dfe 100644 --- a/Untitled.canvas +++ b/.trash/Untitled.canvas diff --git a/Checklist for network node issues after rebooting.md b/Checklist for network node issues after rebooting.md new file mode 100644 index 0000000..013f1dc --- /dev/null +++ b/Checklist for network node issues after rebooting.md @@ -0,0 +1,18 @@ +--- +tags: + - groupvps + - ovn +--- +Check if links are up `ip l` +Check contents of `/etc/network/interfaces` as well as the files it sources. +Check if we have routes `ip r +``` +[jasras@n01.network.vps1-cph8 ~]$ ip r +default via 10.67.1.1 dev bond0 mtu 1500 +10.67.1.0/24 dev bond0 proto kernel scope link src 10.67.1.41 +10.67.3.0/24 dev bond0.21 proto kernel scope link src 10.67.3.41 +10.67.4.0/24 dev bond0.22 proto kernel scope link src 10.67.4.41 +10.67.5.0/24 dev bond0.23 proto kernel scope link src 10.67.5.41 +10.68.0.0/20 via 10.68.1.1 dev bond0.300 +10.68.1.0/24 dev bond0.300 proto kernel scope link src 10.68.1.41 +```
\ No newline at end of file diff --git a/Custom OVN debian packaging.md b/Custom OVN debian packaging.md new file mode 100644 index 0000000..a895469 --- /dev/null +++ b/Custom OVN debian packaging.md @@ -0,0 +1,7 @@ +--- +tags: + - groupvps +--- +There was an issue with unit files and the files in /etc/default being prefixed with group.one-ovn. We want to make sure that the contents of our debian package are identical to regular OVN packages. + +Fixed by using overrides in debian/rules to rename those files after they're installed.
\ No newline at end of file diff --git a/How to see over which network nodes traffic is being routed.md b/How to see over which network nodes traffic is being routed.md new file mode 100644 index 0000000..d38e9b8 --- /dev/null +++ b/How to see over which network nodes traffic is being routed.md @@ -0,0 +1,20 @@ +--- +tags: + - groupvps + - network +--- +In group yaml zie external networks en check aan welk netwerk router port hangt + +ssh netwerk node ( n01 bijv ) + +ovn-nbctl show | grep -C5 gateway + +elke router lrp gateway chassis' + +sbctl list + + +openstack routers met bgp announcen na upgrades, + + +ovn-controller, ovsdb, ovs-vswitchd
\ No newline at end of file diff --git a/OVN Clusters.md b/OVN Clusters.md deleted file mode 100644 index 2bed4f3..0000000 --- a/OVN Clusters.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -tags: - - work - - groupvps ---- -Created the following inventory and put it in ansible for the [[OVN upgrade]] -# CPH8 -``` -9cf3 (9cf3 at tcp:10.67.1.41:6644) (self) -d9b2 (d9b2 at tcp:10.67.9.41:6644) -6c0f (6c0f at tcp:10.67.1.42:6644) last msg 127 ms ago -``` -n01.network.vps1: 10.67.1.41 -n02.network.vps1: 10.67.1.42 -n01.network.vps2: 10.67.9.41 - - -vps1: 10.67.1.0/24 -vps2: 10.67.9.0/24 -vps3: 10.67.17.0/24 - -# LEJ1 -``` -6f0a (6f0a at tcp:10.69.9.41:6643) next_index=6378278 match_index=6378277 last msg 163 ms ago -6e42 (6e42 at tcp:10.69.1.41:6643) (self) next_index=6373918 match_index=6378277 -cec5 (cec5 at tcp:10.69.1.42:6643) next_index=6378278 match_index=6378277 last msg 163 ms ago -3f04 (3f04 at tcp:10.69.17.41:6643) next_index=6378278 match_index=6378277 last msg 163 ms ago - -``` -n01.network.vps1: 10.69.1.41 -n02.network.vps1: 10.69.1.42 -n01.network.vps2: 10.69.9.41 -n01.network.vps3: 10.69.17.41 - -vps1: 10.69.1.0/24 -vps2: 10.69.9.0/24 -vps3: 10.69.17.0/24 -# Freenet -``` -66b0 (66b0 at tcp:10.69.25.41:6643) (self) next_index=426770 match_index=433645 -0a3f (0a3f at tcp:10.69.33.41:6643) next_index=433646 match_index=433645 last msg 85 ms ago -8739 (8739 at tcp:10.69.41.41:6643) next_index=433646 match_index=433645 last msg 85 ms ago -``` -n01.network.vps1: 10.69.25.41 -n01.network.vps2: 10.69.33.41 -n01.network.vps3: 10.69.41.41 - -vps1: 10.69.25.0/24 -vps2: 10.69.33.0/24 -vps3: 10.69.41.0/24 -# Testpod -``` -5627 (5627 at tcp:10.64.101.4:6643) (self) next_index=452726 match_index=456341 -22de (22de at tcp:10.64.101.3:6643) next_index=456342 match_index=456341 last msg 280 ms ago -aed8 (aed8 at tcp:10.64.101.26:6643) next_index=456342 match_index=456341 last msg 280 ms ago -``` -n01.network: 10.64.101.4 -n01.compute: 10.64.101.3 -- [ ] n02.compute: 10.64.101.26
\ No newline at end of file diff --git a/OVN upgrade playbook.md b/OVN upgrade playbook.md new file mode 100644 index 0000000..c305325 --- /dev/null +++ b/OVN upgrade playbook.md @@ -0,0 +1,29 @@ +--- +tags: + - groupvps +--- +1. Test failover. + 1. Start an xping + 2. Log onto n01 network vps4 + 3. Stop ovn-controller, ovs-vswitchd, ovsdb-server processes + 4. See what happens to pings + 1. Expected: around 5 lost packets on about 50% of router interfaces + 2. Unexpected: losing more than 20 packets +2. ansible update apt cache +3. ansible playbook 22.03 + + +--- +Rebooted n01 vps4 -> failover success +Started maintance network nodes until vps3 +Stopped maint due to ovn-controller's not reconnecting succesfully +Stopped ovn-northd on n02 network vps1 + +Rutger wakker gebeld +Rutger investigate, ip r, lots of routes missing +Ansible netplan, inventory lacking infra ranges for vps3 and vps4 + +**In de eerste nacht** vond ik het niet zo heel chill om het met Mohammed te doen. Ik toonde zelf wat onzekerheid omdat het voor mij ook iets spannends is en iets wat ik niet vaak gedaan heb. Daardoor werd hij heel enthousiast met "proberen te helpen" wat tot lichte frustratie zorgde bij ons beide omdat ik niet echt meer rustig kon nadenken zonder hem te moeten tegen houden af en toe. Ik was blij zodra Rutger er was dat Mohammed zich wat meer inhield. + +# Interfaces +Check `/etc/network/interfaces`. Make sure that we have routes configured as per our ansible netplan.
\ No newline at end of file diff --git a/OVN upgrade.md b/OVN upgrade.md index 8874ef4..9ae1962 100644 --- a/OVN upgrade.md +++ b/OVN upgrade.md @@ -7,34 +7,12 @@ tags: https://docs.ovn.org/en/latest/intro/install/ovn-upgrades.html **Fail-safe upgrade procedure** -The ansible playbook does **NOT** encode the fail-safe procedure in such a way that it prevents us from skipping versions. It is **mandatory** that we ourselves correctly decide the version to upgrade to. +Pins ovn-controller version. Then upgrades databases followed by ovn controllers. +Used to upgrade between non LTS version or from LTS to non LTS etc. -**Is the issue resolved with group.one- prefixes on unit files and /etc/default?** -https://gitlab.group.one/groupvps/ovn-builder/-/merge_requests/11/diffs +**Rolling-upgrade** +From LTS to the next LTS. +Upgrades ovn controllers first, then upgrade databases. -It seems to have been fixed using overrides in debian/rules that move stuff to the expected places after building. - -*After testing it looks like this is all done correctly now* - - -**Ansible playbook** -https://gitlab.group.one/groupvps/ansible/-/merge_requests/56/diffs - -- Makes a backup of OVN db's on networking nodes - - Creates files on the local system under /tmp - - Creates files on the remote system under /var/.ovn-backups -- Pins the controller version on both compute and networking nodes - - `ovs-vsctl set open_vswitch . external_ids:ovn-match-northd-version=true` -- Upgrades ovn-central, ovn-common and ovn-host on network nodes - - Installs packages - - restarts ovn-controller, ovn-northd, ovn-ovsdb-server-sb/nb - - Checks ovsdb-server is running - - Checks the socket file for existence -- Upgrades ovn controller on compute nodes - - Installs package - - restart ovn-controller - -**Invocation** -``` -ansible-playbook playbooks/failsafe_install_ovn.yml --limit vps_testpod --extra-vars "ovn_search_version=22.03.8+20250403.1251.35813e0b.systems.jammy1" -```
\ No newline at end of file +[[Custom OVN debian packaging]] +[[Checklist for network node issues after rebooting]]
\ No newline at end of file diff --git a/Random notes on Freenet deployment.md b/Random notes on Freenet deployment.md index e028053..60c2fa0 100644 --- a/Random notes on Freenet deployment.md +++ b/Random notes on Freenet deployment.md @@ -3,6 +3,6 @@ tags: - work - freenet --- -Uses [[Overview of Ceph]] as a volume backing store. Mehmet Top is the primary contact for this Ceph deployment. + Uses [[Overview of Ceph]] as a volume backing store. Mehmet Top is the primary contact for this Ceph deployment. Their mbox VM's also connect to Quobyte storage.
\ No newline at end of file diff --git a/Reading a file line-by-line in Python is easy with a for loop.md b/Reading a file line-by-line in Python is easy with a for loop.md new file mode 100644 index 0000000..d011e91 --- /dev/null +++ b/Reading a file line-by-line in Python is easy with a for loop.md @@ -0,0 +1,10 @@ +--- +tags: + - python +--- +An easy and efficient way to read a file line-by-line is by simple using the file object as the iterable. + +``` +for line in file: + print(line) +``` diff --git a/Systemd timer.md b/Systemd timer.md new file mode 100644 index 0000000..72f50b8 --- /dev/null +++ b/Systemd timer.md @@ -0,0 +1,45 @@ +--- +tags: + - systemd +--- +Managed similarly to a unit. So we can start/stop/enable them etc. + +List: `systemctl list-timers [--all]` + +**Realtime Timers**; same as cron. Use `OnCalendar` +**Monotonic Timers**; Varying start point, then interval. E.g `OnBootSec` specifies start x seconds after boot. + +Each `foo.timer` has a `foo.service` which it calls. Service doesn't need install section. Timer can call different unit by `[Timer] Unit=` + +# Time span format +usec, us, μs +msec, ms +seconds, second, sec, s +minutes, minute, min, m +hours, hour, hr, h +days, day, d +weeks, week, w +months, month, M (defined as 30.44 days) +years, year, y (defined as 365.25 days) + +# OnCalendar Format +`DayOfWeek Year-Month-Day Hour:Minute:Second` + +``` +OnCalendar=Mon,Tue *-*-01..04 12:00:00 +OnCalendar=Sat *-*-1..7 18:00:00 +OnCalendar=*-*-* 4:00:00 +OnCalendar=Mon..Fri 22:30 +OnCalendar=Sat,Sun 20:00 +OnCalendar=*-*-* 02:00:00 Europe/Paris +``` + + + + + + +--- +https://wiki.archlinux.org/title/Systemd/Timers +https://man.archlinux.org/man/systemd.time.7 +https://man.archlinux.org/man/systemd.timer.5
\ No newline at end of file diff --git a/Using Ansible wait_for to wait until a file exists.md b/Using Ansible wait_for to wait until a file exists.md new file mode 100644 index 0000000..7231195 --- /dev/null +++ b/Using Ansible wait_for to wait until a file exists.md @@ -0,0 +1,11 @@ +--- +tags: + - ansible +--- +Using the wait_for module we can wait for a file to exist. + +``` +ansible.builtin.wait_for: + path: /tmp/fo + timeout: 600 +```
\ No newline at end of file |