diff options
author | Jasper Ras <jras@hostnet.nl> | 2025-03-20 11:07:49 +0100 |
---|---|---|
committer | Jasper Ras <jras@hostnet.nl> | 2025-03-20 11:07:49 +0100 |
commit | 80ccf68f55dbb70d7e5ed52ee95b3c9d1b6ce264 (patch) | |
tree | 93e28e85ab70052aa6f577998ec7dc1f413b40c0 /3 Resources | |
parent | 9642cd7ae24f0ba79ce5647c709b35ae8f06a285 (diff) |
vault backup: 2025-03-20 11:07:48
Diffstat (limited to '3 Resources')
57 files changed, 778 insertions, 6 deletions
diff --git a/3 Resources/Bash.md b/3 Resources/Bash.md deleted file mode 100644 index 841443f..0000000 --- a/3 Resources/Bash.md +++ /dev/null @@ -1 +0,0 @@ -How to [[Reference arguments]].
\ No newline at end of file diff --git a/3 Resources/Bash/Expansion.md b/3 Resources/Bash/Expansion.md new file mode 100644 index 0000000..1cd3209 --- /dev/null +++ b/3 Resources/Bash/Expansion.md @@ -0,0 +1,9 @@ +--- +tags: + - bash +--- +Find about in the manual `man bash` (search EXPANSION). This document will hold some often used expansions. + +**Variable expansion** +substr +`${VAR:offset:len}`
\ No newline at end of file diff --git a/3 Resources/Bash/Generate random integers.md b/3 Resources/Bash/Generate random integers.md new file mode 100644 index 0000000..8835716 --- /dev/null +++ b/3 Resources/Bash/Generate random integers.md @@ -0,0 +1,19 @@ +--- +tags: + - bash +--- +The environment variable `RANDOM` contains an int between 0-32768. + +``` +min=10 +max=20 +echo $(($RANDOM%($max-$min+1)+$min)) +20 +echo $(($RANDOM%($max-$min+1)+$min)) +14 +``` + +Using parameter [[Expansion]] we can just grab the first number: +``` +${RANDOM:0} +```
\ No newline at end of file diff --git a/3 Resources/Bash/Heredoc.md b/3 Resources/Bash/Heredoc.md new file mode 100644 index 0000000..703e3e5 --- /dev/null +++ b/3 Resources/Bash/Heredoc.md @@ -0,0 +1,16 @@ +``` +read -r -d '' VAR <<EOF +blaat +EOF +``` + +``` +var=$( +cat <<EOF +blaat +EOF +) +``` + +Quote EOF/NAME to prevent expansion +Put - after << to remove indentation from result
\ No newline at end of file diff --git a/3 Resources/Dictionary/Scalar type.md b/3 Resources/Dictionary/Scalar type.md new file mode 100644 index 0000000..3cd22be --- /dev/null +++ b/3 Resources/Dictionary/Scalar type.md @@ -0,0 +1,13 @@ +--- +tags: + - programming + - dictionary +--- +A scalar type represents a single value. Thus +- int +- float +- string +- double +- boolean + +are all scalar types ( even though they can have a class such as in python or rust! ).
\ No newline at end of file diff --git a/3 Resources/Dictionary/Type inference.md b/3 Resources/Dictionary/Type inference.md new file mode 100644 index 0000000..7b8d8e1 --- /dev/null +++ b/3 Resources/Dictionary/Type inference.md @@ -0,0 +1,17 @@ +--- +tags: + - programming + - dictionary +--- +This is when the compiler can infer (determine) the type of a variable by looking at the value being assigned. + +For example: +```rust +let x = 1 + +# Equals + +let x: u32 = 1; +``` + +But the Rust compiler infers that x is u32 because that is the default int type.
\ No newline at end of file diff --git a/3 Resources/Git/Edit existing commits.md b/3 Resources/Git/Edit existing commits.md new file mode 100644 index 0000000..9832ee5 --- /dev/null +++ b/3 Resources/Git/Edit existing commits.md @@ -0,0 +1,22 @@ +--- +tags: + - git +--- +I've found a nice and easy way to edit existing commits. Before I used the following "tedious" process. +``` +git stash # Git rebase wants a clean working tree +git rebase -i <main|master|whatever branch your branched off> +... in the interactive editor, find the commit to edit, and mark it to edit ... +... write & quit ... +git stash pop # Optional. Only if you've already made the changes on HEAD. +git commit --amend +git rebase --continue +``` + +Now we can do something much easier using a so called "fixup commit". + +``` +git commit --fixup=<commit_id_to_fix> +git rebase -i --autosquash <main|master|whatever branch you branched off> +... write & quit +```
\ No newline at end of file diff --git a/3 Resources/Git/Find files in history.md b/3 Resources/Git/Find files in history.md new file mode 100644 index 0000000..d5592fe --- /dev/null +++ b/3 Resources/Git/Find files in history.md @@ -0,0 +1,17 @@ +--- +tags: + - git +references: + - https://stackoverflow.com/questions/7203515/how-to-find-a-deleted-file-in-the-project-commit-history +--- +Find a (deleted) file in all commits. +`g lg --all --full-history -- "**/shell.nix"` +or more generally +`g lg --all --full-history -- <path-to-file>` +then using the SHA +`g show <sha> -- <path-to-file>` + +**Epic bonus** +Restore the file: +`g checkout <sha>^ -- <path-to-file>` +the caret (`^`) references one commit before sha, because in sha it is deleted.
\ No newline at end of file diff --git a/3 Resources/Index/Bash.md b/3 Resources/Index/Bash.md new file mode 100644 index 0000000..cee90e3 --- /dev/null +++ b/3 Resources/Index/Bash.md @@ -0,0 +1,8 @@ +--- +tags: + - bash +--- +* [[Reference arguments]]. +* [[Generate random integers]] +* [[Expansion]] +* [[Heredoc]]
\ No newline at end of file diff --git a/3 Resources/Index/Nix.md b/3 Resources/Index/Nix.md new file mode 100644 index 0000000..60a8e40 --- /dev/null +++ b/3 Resources/Index/Nix.md @@ -0,0 +1,8 @@ +--- +tags: + - nix + - index +--- +[[Interpolation]] +[[NixOS Modules]] +[[Shell]]
\ No newline at end of file diff --git a/3 Resources/Index/jq.md b/3 Resources/Index/jq.md new file mode 100644 index 0000000..07d43aa --- /dev/null +++ b/3 Resources/Index/jq.md @@ -0,0 +1,7 @@ +--- +tags: + - jq +references: + - https://jqlang.org/ +--- +[[about]] diff --git a/3 Resources/Linux/BatchMode.md b/3 Resources/Linux/BatchMode.md new file mode 100644 index 0000000..e45de5a --- /dev/null +++ b/3 Resources/Linux/BatchMode.md @@ -0,0 +1,6 @@ +--- +tags: + - ssh + - linux +--- +With BatchMode if SSH will fail instantly if the key is rejected, rather than prompting for a password and causing scripts to hang indefinitely.
\ No newline at end of file diff --git a/3 Resources/Linux/Create bootable USB.md b/3 Resources/Linux/Create bootable USB.md new file mode 100644 index 0000000..a296677 --- /dev/null +++ b/3 Resources/Linux/Create bootable USB.md @@ -0,0 +1,8 @@ +--- +tags: + - linux + - bootable +--- +``` +sudo dd bs=4M conv=fsync oflag=direct status=progress if=linux.iso of=/dev/sda +```
\ No newline at end of file diff --git a/3 Resources/Linux/Device mapper.md b/3 Resources/Linux/Device mapper.md index 059a0d1..27980ca 100644 --- a/3 Resources/Linux/Device mapper.md +++ b/3 Resources/Linux/Device mapper.md @@ -4,7 +4,6 @@ Device mapper is a kernel driver to map physical block devices to virtual ones; this means that for example we can have many physical devices be represented by a single virtual one (linear mapping). How does it work? -[[1 projects/Inwerken Mohammed/TODO]] What is its function in relation to multipath? Logical devices using device-mapper can be managed using `man 8 dmsetup` diff --git a/3 Resources/Linux/Exit SSH control master connection.md b/3 Resources/Linux/Exit SSH control master connection.md new file mode 100644 index 0000000..7054d9d --- /dev/null +++ b/3 Resources/Linux/Exit SSH control master connection.md @@ -0,0 +1,6 @@ +--- +tags: + - ssh + - linux +--- +To exit an existing session use `ssh -O exit <user>@<host>`
\ No newline at end of file diff --git a/3 Resources/Linux/filesystems/Tmpfs.md b/3 Resources/Linux/FS/Tmpfs.md index b8fe839..b8fe839 100644 --- a/3 Resources/Linux/filesystems/Tmpfs.md +++ b/3 Resources/Linux/FS/Tmpfs.md diff --git a/3 Resources/Linux/FS/Writeback cache.md b/3 Resources/Linux/FS/Writeback cache.md new file mode 100644 index 0000000..03917bb --- /dev/null +++ b/3 Resources/Linux/FS/Writeback cache.md @@ -0,0 +1,36 @@ +--- +tags: + - linux + - qemu + - filesystem +references: + - https://avidandrew.com/understanding-disk-cache-writeback-ext4.html + - https://docs.kernel.org/admin-guide/sysctl/vm.html#dirty-expire-centisecs +--- +normal: write() -> cache (multiple layers) -> disk + +# Physical server write cache +![[physical_write_cache.png]] + +Page cache -> RAM + +Journal -> ensures data is fully written before the transaction is considered complete. +`commit` mount option -> flushes cache to disk every x seconds (configurable), ==default = 5.== +`barrier` mount option -> enables the ordering of groups of writes, controller ensures writes before barrier are written before writes after barrier. ==Default = 1== + +`commit` + dirty_expire_centisecs [2] ~ automatic persisting of data. + +If we call `sync`, `fsync` or `fdatasync` ourselves our data is forced on-disk right away by the kernel, no need to wait for commit + dirty_expirty_centisecs. + +# VM Write cache + Guest has its own page cache + ![[vm_write_cache.png]] + +QEMU/KVM options for disk caches: +- Writeback -> write complete if in host page cache; guest required to flush for integrity. +- Writethrough -> write complete if committed to disk; guest no need to flush. +- None -> Equivalent to direct access to host disk. Guest needs to flush. +- Unsafe -> eq writeback but flush ignored. Performance. +- Directsync -> writethrough but without host page cache. + +Safe if guest uses `commit` and `barrier` and therefore `fdatasync` syscalls on a recurring basis.
\ No newline at end of file diff --git a/3 Resources/Linux/FS/physical_write_cache.png b/3 Resources/Linux/FS/physical_write_cache.png Binary files differnew file mode 100644 index 0000000..819c834 --- /dev/null +++ b/3 Resources/Linux/FS/physical_write_cache.png diff --git a/3 Resources/Linux/FS/vm_write_cache.png b/3 Resources/Linux/FS/vm_write_cache.png Binary files differnew file mode 100644 index 0000000..1c63843 --- /dev/null +++ b/3 Resources/Linux/FS/vm_write_cache.png diff --git a/3 Resources/Linux/Grepping stuff.md b/3 Resources/Linux/Grepping stuff.md new file mode 100644 index 0000000..2d6af44 --- /dev/null +++ b/3 Resources/Linux/Grepping stuff.md @@ -0,0 +1,15 @@ +--- +tags: + - grep + - ripgrep + - linux +--- +# Grep +`grep -vxf gobs_servers onecrm_servers` grep all onecrm_servers that **ARE NOT** in gobs_servers +`grep -oxf gobs_servers onecrm_servers` grep all onecrm_servers that **ARE** in gobs_servers +`grep -vxf onecrm_servers gobs_servers` grep all gobs_servers that **ARE NOT** in onecrm_servers +`grep -oxf onecrm_servers gobs_servers` grep all gobs_servers that **ARE** in onecrm_servers +# Ripgrep +Pass `-L` to `rg` to make it follow symlinks: this is handy for example when searching hieradata, because node definitions can be symlinked. + + `-l` for showing only files with matches
\ No newline at end of file diff --git a/3 Resources/Linux/NetworkManager.md b/3 Resources/Linux/NetworkManager.md new file mode 100644 index 0000000..545124c --- /dev/null +++ b/3 Resources/Linux/NetworkManager.md @@ -0,0 +1,43 @@ +--- +tags: + - linux + - networking +references: + - https://networkmanager.dev/ + - https://ubuntu.com/core/docs/networkmanager/configure-vpn + - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/assembly_networkmanager-connection-profiles-in-keyfile-format_configuring-and-managing-networking +--- +Connection profiles (nm-settings-nmcli(5)) +Networkmanager will only read profile files owned by root bc they often contain secrets. +Keyfile format for connection profiles is .INI-like, but makes it possible for example to write a list of arrays in a space-seperated string? + + +> Configuring secrets for when using OpenVPN together with NetworkManager in NixOS is complete dogshit. Avoid :) + +Manually defining VPN +https://networkmanager.dev/docs/api/latest/nm-settings-nmcli.html +``` +nmcli c add connection.id vpntest connection.type vpn \ + vpn.service-type org.freedesktop.NetworkManager.openvpn \ + ipv4.never-default true \ + ipv6.never-default true \ + +vpn.data ca=/sjj:qvar/snap/network-manager/common/creds/server_ca.crt \ + +vpn.data cert=/var/snap/network-manager/common/creds/user.crt \ + +vpn.data cert-pass-flags=0 \ + +vpn.data cipher=AES-128-CBC \ + +vpn.data comp-lzo=adaptive \ + +vpn.data connection-type=tls \ + +vpn.data dev=tun \ + +vpn.data key=/var/snap/network-manager/common/creds/user.key \ + +vpn.data ping=10 \ + +vpn.data ping-restart=60 \ + +vpn.data remote=<server>:<port> \ + +vpn.data remote-cert-tls=server \ + +vpn.data ta=/var/snap/network-manager/common/creds/tls_auth.key \ + +vpn.data ta-dir=1 \ + +vpn.data verify-x509-name=name:access.is +``` + + +Conver nmconnection into Nix +https://github.com/janik-haag/nm2nix
\ No newline at end of file diff --git a/3 Resources/Linux/Use type if which doesnt work.md b/3 Resources/Linux/Use type if which doesnt work.md new file mode 100644 index 0000000..39881dc --- /dev/null +++ b/3 Resources/Linux/Use type if which doesnt work.md @@ -0,0 +1,7 @@ +--- +tags: + - linux + - shell +--- +Sometimes `which` will say: HEY it's a function! +Then we can use `type` :) It even shows the defition!
\ No newline at end of file diff --git a/3 Resources/Linux/Useful info about Debian packages.md b/3 Resources/Linux/Useful info about Debian packages.md new file mode 100644 index 0000000..f401790 --- /dev/null +++ b/3 Resources/Linux/Useful info about Debian packages.md @@ -0,0 +1,9 @@ +--- +tags: + - debian + - packaging + - linux +--- +https://www.debian.org/doc/debian-policy/index.html + +Specifically https://www.debian.org/doc/debian-policy/ch-relationships.html
\ No newline at end of file diff --git a/3 Resources/Linux/Using GRUB we need an EF02 partition.md b/3 Resources/Linux/Using GRUB we need an EF02 partition.md new file mode 100644 index 0000000..8fd3109 --- /dev/null +++ b/3 Resources/Linux/Using GRUB we need an EF02 partition.md @@ -0,0 +1,13 @@ +--- +tags: + - linux + - grub +--- +I've learnt that to install GRUB we need an EF02 partition as the first partition. + +MBR -> first sector of disk contains boot loader +GPT -> a seperate partition + +EF00 -> EFI System Partition (ESP) +EF02 -> BIOS boot partition +Both are partitions for GPT partitioned disks, just the type of firmware is different,.
\ No newline at end of file diff --git a/3 Resources/math/Cartesian product.md b/3 Resources/Math/Cartesian product.md index 161b405..161b405 100644 --- a/3 Resources/math/Cartesian product.md +++ b/3 Resources/Math/Cartesian product.md diff --git a/3 Resources/NFS.md b/3 Resources/NFS.md new file mode 100644 index 0000000..20edb9f --- /dev/null +++ b/3 Resources/NFS.md @@ -0,0 +1,32 @@ +--- +tags: + - linux + - nfs +references: + - https://linux.die.net/man/5/exports + - https://wiki.gentoo.org/wiki/Nfs-utils + - https://docs.freebsd.org/en/books/handbook/network-servers/#network-nfs +--- +# Server +daemons: +- nfsd : handles reqs from nfs clients +- mountd : handles reqs from nfsd +- rpcbind : allows nfs clients to discover which port the NFS server is using +# Exports +`/etc/exports` contains table of FS' that are exported over NFS. Newline-seperated list of exports that are in turn whitespace seperated with clients allowed to mount there. A client can be followed by parenthesised comma-seperated list of export opts. +Default opts can be specified at the end behind a dash. + +**Client specs** +- Single host (name or ip) +- Netgroups +- Wildcards (`?` or `*` and `[]` char classes) +- IP networks (`ipaddr/netmask` ) + +**Some useful options** +- rw +- ro + +**Example** +``` +/nfs 192.168.50.0/24(rw,sync,insecure,no_subtree_check) +```
\ No newline at end of file diff --git a/3 Resources/Nix/Adding things to the nix store.md b/3 Resources/Nix/Adding things to the nix store.md new file mode 100644 index 0000000..98b2c17 --- /dev/null +++ b/3 Resources/Nix/Adding things to the nix store.md @@ -0,0 +1,12 @@ +--- +tags: + - nix +--- +It's possible to use `nix-prefetch-url`: +``` +nixos main +❯ nix-prefetch-url --name displaylink-610.zip https://www.synaptics.com/sites/default/files/exe_files/2024-10/DisplayLink%20USB%20Graphics%20Software%20for%20Ubuntu6.1-EXE.zip +path is '/nix/store/a10lxg1y9hc5ida4npg2mrmymc8932hl-displaylink-610.zip' +1b3w7gxz54lp0hglsfwm5ln93nrpppjqg5sfszrxpw4qgynib624 + +```
\ No newline at end of file diff --git a/3 Resources/Nix/Cleaning the Nix store.md b/3 Resources/Nix/Cleaning the Nix store.md new file mode 100644 index 0000000..4ec07c0 --- /dev/null +++ b/3 Resources/Nix/Cleaning the Nix store.md @@ -0,0 +1,11 @@ +--- +tags: + - nix +references: + - https://www.reddit.com/r/NixOS/comments/10107km/how_to_delete_old_generations_on_nixos/ + - https://nixos.org/manual/nixos/unstable/#sec-nix-gc +--- +``` +sudo nix-collect-garbage -d +sudo nixos-rebuild boot +```
\ No newline at end of file diff --git a/3 Resources/Nix/Flake.md b/3 Resources/Nix/Flake.md index 754556f..d191989 100644 --- a/3 Resources/Nix/Flake.md +++ b/3 Resources/Nix/Flake.md @@ -18,5 +18,67 @@ In flakes dependencies have to be specified explicitly and MUST be locked to spe Output of a Flake is an arbitrary [[Zettelkast/Index/Nix]] value such as a package, [[NixOS]] module or library function. Commands `nix build` and `nix shell` will build the output `packages.<system>.default` unless we specify another output, for example: `nix shell .#checks.aarch64-linux.build`. + +**Inputs** +``` +{ + inputs = { + # GitHub repository as the data source, specifying the master branch. + # This is the most common input format. + nixpkgs.url = "github:Mic92/nixpkgs/master"; + # Git URL, applicable to any Git repository using the https/ssh protocol. + git-example.url = "git+https://git.somehost.tld/user/path?ref=branch"; + # Git URL by tag, applicable to any Git repository using the https/ssh protocol. + git-example-tag.url = "git+https://git.somehost.tld/user/path?tag=x.y.x"; + # Github URL by pull request. + git-pr.url = "github:NixOS/nixpkgs?ref=pull/349351/head"; + # Git URL with submodules, applicable to any Git repository using the https/ssh protocol. + git-example-submodule.url = "git+https://git.somehost.tld/user/path?submodules=1"; + # Archive File URL, needed in case your input use LFS. + # Regular git input doesn't support LFS yet. + git-example-lfs.url = "https://codeberg.org/solver-orgz/treedome/archive/master.tar.gz"; + # Similar to fetching a Git repository, but using the ssh protocol + # with key authentication. Also uses the shallow=1 parameter + # to avoid copying the .git directory. + ssh-git-example.url = "git+ssh://git@github.com/ryan4yin/nix-secrets.git?shallow=1"; + # It's also possible to directly depend on a local Git repository. + git-directory-example.url = "git+file:/path/to/repo?shallow=1"; + # Using the `dir` parameter to specify a subdirectory. + nixpkgs.url = "github:foo/bar?dir=shu"; + # Local folder (if using an absolute path, the 'path:' prefix can be omitted). + directory-example.url = "path:/path/to/repo"; + + # If the data source is not a flake, set flake=false. + # `flake=false` is usually used to include additional source code, + # configuration files, etc. + # In Nix code, you can directly reference files within + # it using "${inputs.bar}/xxx/xxx" notation. + # For example, import "${inputs.bar}/xxx/xxx.nix" to import a specific nix file, + # or use "${inputs.bar}/xx/xx" as a path parameter for certain options. + bar = { + url = "github:foo/bar/branch"; + flake = false; + }; + + sops-nix = { + url = "github:Mic92/sops-nix"; + # `follows` is the inheritance syntax within inputs. + # Here, it ensures that sops-nix's `inputs.nixpkgs` aligns with + # the current flake's inputs.nixpkgs, + # avoiding inconsistencies in the dependency's nixpkgs version. + inputs.nixpkgs.follows = "nixpkgs"; + }; + + # Lock the flake to a specific commit. + nix-doom-emacs = { + url = "github:vlaci/nix-doom-emacs?rev=238b18d7b2c8239f676358634bfb32693d3706f3"; + flake = false; + }; + }; + + outputs = { self, ... }@inputs: { ... }; +} +``` + --- [Flakes Wiki](https://nixos.wiki/wiki/Flakes)
\ No newline at end of file diff --git a/3 Resources/Nix/Functional programming vs imperative.md b/3 Resources/Nix/Functional programming vs imperative.md new file mode 100644 index 0000000..8c2619a --- /dev/null +++ b/3 Resources/Nix/Functional programming vs imperative.md @@ -0,0 +1,26 @@ +--- +tags: + - programming + - nix +--- +While working on a NixOS module I made a realisation on the difference between functional and imperative programming. +I was trying to configure `systemd.tmpfiles.rules` to create directories for git repositories. After looking a bit how to iterate; i tried the following: +```nix +let + paths = ["abs_paths" ...] +in +{ + map(p: systemd.tmpfiles.rules = ["d {$p} ..."];); +} +``` +This is very much originating from the imperative mindset; we loop over a list and then do stuff like setting variables and calling other functions. However in functional programming we do not! This is the correct version in functional programming: +```nix +let + paths = [ "abs_paths" ... ]; +in +{ + systemd.tmpfiles.rules = map(p: "d ${p} ..") paths; +} +``` + +Functions are pure, we can't assign stuff inside of their bodies because that would make them impure. Instead we can just return a value; in this case an array and *then* assign it.
\ No newline at end of file diff --git a/3 Resources/Nix/Generating Nix expressions.md b/3 Resources/Nix/Generating Nix expressions.md new file mode 100644 index 0000000..347a7af --- /dev/null +++ b/3 Resources/Nix/Generating Nix expressions.md @@ -0,0 +1,7 @@ +--- +tags: + - nix +--- +From this repo it looks like we can convert JSON into Nix using `nix-instantiate --expr --eval builtins.fromJSON <jsonfile> ` + +https://github.com/Janik-Haag/nm2nix/blob/main/nm2nix.py
\ No newline at end of file diff --git a/3 Resources/Nix/Home-manager systemd.md b/3 Resources/Nix/Home-manager systemd.md new file mode 100644 index 0000000..d4a1460 --- /dev/null +++ b/3 Resources/Nix/Home-manager systemd.md @@ -0,0 +1,35 @@ +--- +tags: + - nix + - home-manager + - systemd +--- +Config options start with `systemd.user` + +Home-manager manages the enabling and such based on the dependencies of the unit. +So we can something like +``` +Install = { + WantedBy = [ "default.target" ]; +}; +``` + +And it will enable it. + +Another example +```nix +systemd.user.services.astal = { + Unit = { + Description = "Runs the astal bar instance"; + After = [ "graphical-session-pre.target" ]; + }; + Service = { + Type = "exec"; + ExecStart = "some-bin"; + }; + + Install = { + WantedBy = [ "graphical-session.target" ]; + }; +}; +```
\ No newline at end of file diff --git a/3 Resources/Nix/Hostname changes.md b/3 Resources/Nix/Hostname changes.md new file mode 100644 index 0000000..0aaa324 --- /dev/null +++ b/3 Resources/Nix/Hostname changes.md @@ -0,0 +1,6 @@ +--- +tags: + - nixos + - nixos-anywhere +--- +When using nixos-anywhere and passing it a hostname; it changes during kexec to nixos-anywhere. But the script will continue attempting to connect to whatever we gave it. Therefore we should pass it an IP address instead.
\ No newline at end of file diff --git a/3 Resources/Nix/Interpolation.md b/3 Resources/Nix/Interpolation.md new file mode 100644 index 0000000..bbdb457 --- /dev/null +++ b/3 Resources/Nix/Interpolation.md @@ -0,0 +1,7 @@ +--- +tags: + - nix +references: + - https://nix.dev/manual/nix/2.24/language/syntax#string-literal +--- +In a normal string we can use backslash to escape. In indented strings we should use double backticks.
\ No newline at end of file diff --git a/3 Resources/Nix/Nix Manual References.md b/3 Resources/Nix/Nix Manual References.md new file mode 100644 index 0000000..486a2fe --- /dev/null +++ b/3 Resources/Nix/Nix Manual References.md @@ -0,0 +1,7 @@ +--- +tags: + - nix +references: + - https://nix.dev/manual/nix/2.24/language/builtins.html + - https://nixos.org/manual/nixpkgs/stable +--- diff --git a/3 Resources/Nix/Nix collect garbage.md b/3 Resources/Nix/Nix collect garbage.md new file mode 100644 index 0000000..65e6aad --- /dev/null +++ b/3 Resources/Nix/Nix collect garbage.md @@ -0,0 +1,10 @@ +--- +tags: + - nix +references: + - https://www.reddit.com/r/NixOS/comments/10107km/how_to_delete_old_generations_on_nixos/ +--- +``` +sudo nix-collect-garbage -d +sudo nixos-rebuild boot +```
\ No newline at end of file diff --git a/3 Resources/Nix/NixOS Modules.md b/3 Resources/Nix/NixOS Modules.md new file mode 100644 index 0000000..1ab4c25 --- /dev/null +++ b/3 Resources/Nix/NixOS Modules.md @@ -0,0 +1,56 @@ +--- +tags: + - nix + - nixos +references: + - https://nixos.org/manual/nixos/unstable/#sec-writing-modules + - https://nixos.org/manual/nixpkgs/stable/#module-system-lib-evalModules + - https://nix.dev/tutorials/module-system/ + - llk +--- +Extra care must be taken when writing systemd services using Exec* due to interpolation and such. `utils.escapeSystemdExecArg` and `utils.escapeSystemdExecArg` exist. + +```nix +{ + options = { + name = mkOption { + type = type specification; + default = default value; + example = example value; + description = "Description for use in the NixOS manual."; + }; + }; +} +``` + +**A list of submodules** +```nix +{ + options.mod = mkOption { + description = "submodule example"; + type = with types; listOf (submodule { + options = { + foo = mkOption { + type = int; + }; + bar = mkOption { + type = str; + }; + }; + }); + }; +} +``` + + +**Testing** +```nix eval.nix +let + pkgs = import <nixpkgs> {}; + res = pkgs.lib.evalModules { + modules = [./git-repos.nix]; + }; +in +res.config +``` +`nix-instantiate --eval eval.nix`
\ No newline at end of file diff --git a/3 Resources/Nix/Remote nixos rebuild lacking valid signature.md b/3 Resources/Nix/Remote nixos rebuild lacking valid signature.md new file mode 100644 index 0000000..82e03b2 --- /dev/null +++ b/3 Resources/Nix/Remote nixos rebuild lacking valid signature.md @@ -0,0 +1,13 @@ +--- +tags: + - nix + - nixos +references: + - https://nixos.wiki/wiki/Nixos-rebuild +--- +When rebuilding on a remote it might spew an error containing: "is lacking a valid signature". To remedy this we need to add our non-root user as a trusted user; +``` +nix.extraOptions = '' + trusted-users jras +''; +```
\ No newline at end of file diff --git a/3 Resources/Nix/Shell.nix.md b/3 Resources/Nix/Shell.md index 13715c2..08e7c53 100644 --- a/3 Resources/Nix/Shell.nix.md +++ b/3 Resources/Nix/Shell.md @@ -1,6 +1,35 @@ -#nix #shell - --- +tags: + - nix + - shell +--- +Basic invocation: `nix-shell -p [ pkgs ... ]` + +# Shebang +``` +#! /usr/bin/env nix-shell +#! nix-shell -i bash -p bash + +do stuff +``` + +Different interpreters can be set with the `-i` option such as python +``` +#! /usr/bin/env nix-shell +#! nix-shell -i python3 -p python3 + +print("hello world") +``` + +The `-I` option can be used to pin nixpkgs +``` +#! /usr/bin/env nix-shell +#! nix-shell -i bash +#! nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/aed4b19d312525ae7ca9bceb4e1efe3357d0e2eb.tar.gz + +echo hello world +``` +# Shell.nix Can be used to set up per dir environments, e.g using direnv to automatically activate them. ## Python https://nixos.org/manual/nixpkgs/stable/#python @@ -38,5 +67,5 @@ let in python311.withPackages (ps: with ps; [ numpy my_toolz ]) ).env ``` -The [[Import]] is required here because imports a nix expression from another source, in this case nixpkgs. + The [[Import]] is required here because imports a nix expression from another source, in this case nixpkgs. It is not required at line two, because python310Packages itself is already in the local scope.
\ No newline at end of file diff --git a/3 Resources/Programming/Integer types Rust.png b/3 Resources/Programming/Integer types Rust.png Binary files differnew file mode 100644 index 0000000..358eb85 --- /dev/null +++ b/3 Resources/Programming/Integer types Rust.png diff --git a/3 Resources/Programming/Rust.md b/3 Resources/Programming/Rust.md new file mode 100644 index 0000000..8724ca3 --- /dev/null +++ b/3 Resources/Programming/Rust.md @@ -0,0 +1,27 @@ +https://rust-book.cs.brown.edu + +> Finished chapter 2. Onto chapter 3! + +If the name ends with ! it's a macro. + +By default, Rust has a set of items defined in the standard library that it brings into the scope of every program. This set is called the _prelude_, and you can see everything in it [in the standard library documentation](https://doc.rust-lang.org/std/prelude/index.html). + +"he `stdin` function returns an instance of [`std::io::Stdin`](https://doc.rust-lang.org/std/io/struct.Stdin.html), which is a type that represents a handle to the standard input for your terminal." +How does this work? How does it know which terminal is ours? I guess it might be connected by linux when running the process as `/proc/<pid>/stdin`. + +`&` is a reference. References are also immutable by default, thus `&mut guess` will allow us to update the reference to the variable guess. + +Enum members are called variants. + +`cargo doc --open` command will build documentation provided by all your dependencies locally and open it in your browser. + +`match` expression -> arms -> pattern + code path. Match expressions stop evaluating if a match is made; therefore we should strive to order them based on their likeliness to be hit. + +`shadowed` = when a variable name is used multiple times; e.g +``` +let x = 1 +let x = 2 <- shadowed +``` +Shadowing is different from making a variable `mutable` because it is fully reassigned (thus its type can also be changed), but the value is then still immutable (giving compiler errors when we try to reassign). + +![[Integer types Rust.png]]
\ No newline at end of file diff --git a/3 Resources/Python/Make dataclass dict-able.md b/3 Resources/Python/Make dataclass dict-able.md index d8ee02b..7cb391b 100644 --- a/3 Resources/Python/Make dataclass dict-able.md +++ b/3 Resources/Python/Make dataclass dict-able.md @@ -12,4 +12,9 @@ We can use `vars` to make that easy: def __iter__(self): for k, v in vars(self): yield k, v +``` + +> This only works when all vars are primtiives, a list of objects for example needs nested iteration. Example: +``` +yield "volumes", [iter(vol) for vol in self.volumes] ```
\ No newline at end of file diff --git a/3 Resources/Python/Python dunder.md b/3 Resources/Python/Python dunder.md new file mode 100644 index 0000000..ca24307 --- /dev/null +++ b/3 Resources/Python/Python dunder.md @@ -0,0 +1,6 @@ +--- +tags: + - python +--- +Everything in python is an object, even strings, functions etc. +all operations are defined as dunder methods; eg "bla" + "bla" == str.\_\_add\_\_ diff --git a/3 Resources/SQLAlchemy/2.0 query style operator reference.md b/3 Resources/SQLAlchemy/2.0 query style operator reference.md new file mode 100644 index 0000000..5161d1c --- /dev/null +++ b/3 Resources/SQLAlchemy/2.0 query style operator reference.md @@ -0,0 +1,7 @@ +--- +tags: + - sqlalchemy +references: + - https://docs.sqlalchemy.org/en/20/core/operators.html +--- +The referenced link is very useful when working with the 2.0-style query. For example, i was figuring out how to add a IN clause to a query and it's documented in there (`in_`).
\ No newline at end of file diff --git a/3 Resources/System feedback.md b/3 Resources/System feedback.md new file mode 100644 index 0000000..97b97a1 --- /dev/null +++ b/3 Resources/System feedback.md @@ -0,0 +1,9 @@ +--- +tags: + - system-design + - software-design + - design +--- +I read an article somewhere about control plane feedback: any action we take should give (direct) feedback. + +This idea can be extended to a control panel implementation, e.g an openstack control panel. Where any action you do provides you with instant feedback. Think of creating a server; but it fails somewhere down the line due to an IP being allocated already. We want to know that so we can try again with another IP.
\ No newline at end of file diff --git a/3 Resources/Systemd/Systemd restart signal.md b/3 Resources/Systemd/Systemd restart signal.md new file mode 100644 index 0000000..3497069 --- /dev/null +++ b/3 Resources/Systemd/Systemd restart signal.md @@ -0,0 +1,25 @@ +--- +tags: + - systemd + - restart +references: + - https://www.reddit.com/r/linuxquestions/comments/10eu3i0/does_systemctl_restart_send_sigterm_signal/?rdt=64556 +--- +By default, `restart` does exactly the same thing as `stop` followed by `start`. + +`stop` will: + +- execute `ExecStop=` commands, if any; + +- send the signal defined by `KillSignal=` (default: `SIGTERM`), to any remaining processes identified by the `KillMode=`; + +- send `SIGCONT` to those same processes; + +- if `SendSIGHUP=` is true (default: false), send `SIGHUP` to those same processes; + +- if `SendSIGKILL=` is true (default: true), and any processes remain after `TimeoutStopSec=`, send the `FinalKillSignal=` (default: `SIGKILL`) to those remaining processes; + +- execute `ExecStopPost=` commands, if any. + + +`restart` changes this by using `RestartKillSignal=` instead of `KillSignal=`, but if `RestartKillSignal=` is not set it just inherits its value from `KillSignal=`. And, of course, `restart` then performs all the regular `start` actions after everything else. `restart` also differs from `stop` by not flushing the file descriptor store associated with the service.
\ No newline at end of file diff --git a/3 Resources/Systemd/Unit types.md b/3 Resources/Systemd/Unit types.md index 27ee387..51bb40f 100644 --- a/3 Resources/Systemd/Unit types.md +++ b/3 Resources/Systemd/Unit types.md @@ -5,10 +5,13 @@ tags: --- The `Type` directive is specified within the `Service` section. There are 6 different types; - **Simple**: The default if `Type` nor `Busname` are specified. +- **Exec** : similar to simple but waits a bit more for the service to be started successfully - **Forking**: Used when the parent process exits immediately due to it forking. Systemd will know it's still running even though it exited. - **Oneshot**: When the process is short-lived and doesn't need to keep running. Systemd will wait on the process before starting other units. - **Dbus**: The process will be registered on the D-Bus bus. - **Notify**: Systemd will expect a notification from the process to signal it has started up blocking the execution of other units until the signal is received. - **Idle**: Service will not run ufntil all jobs are dispatched. #clarify -https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files#unit-specific-section-directives
\ No newline at end of file +https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files#unit-specific-section-directives + +**Exec** is recommended by `man systemd.service` for long-running services.
\ No newline at end of file diff --git a/3 Resources/Tailscale.md b/3 Resources/Tailscale.md new file mode 100644 index 0000000..6cf82c4 --- /dev/null +++ b/3 Resources/Tailscale.md @@ -0,0 +1,16 @@ +--- +tags: + - tailscale + - vpn +references: + - https://tailscale.com/blog/tailscale-key-management +--- +# About keys +Two types of keys: machine keys and node keys. + +The machine key is tied to a specific machine and is used during pre-authorization (started when it first attempts to join a network). + +When successfully added to a network a node key is generated on the client. The public component is tied to the machine key on the control server. +The node key is also tied to an identity; via the URL that's sent back to the client and then used to login. + +The public node key is then distributed to other nodes it's allowed to communicate with.
\ No newline at end of file diff --git a/3 Resources/Techniques/Long polling.md b/3 Resources/Techniques/Long polling.md new file mode 100644 index 0000000..7cbabd0 --- /dev/null +++ b/3 Resources/Techniques/Long polling.md @@ -0,0 +1 @@ +Poll req client -> server keeps open conn, puts req in queue to be processed later, processor picks up from Q and responds to client.
\ No newline at end of file diff --git a/3 Resources/ansible/Waiting for stuff.md b/3 Resources/ansible/Waiting for stuff.md new file mode 100644 index 0000000..e00bb19 --- /dev/null +++ b/3 Resources/ansible/Waiting for stuff.md @@ -0,0 +1,12 @@ +--- +tags: + - ansible +references: + - https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait_for_module.html +--- +There's a module to wait for all sorts of stuff, e.g a file being created or removed: +```yaml + ansible.builtin.wait_for: + path: /opt/puppetlabs/puppet/cache/state/agent_catalog_run.lock + state: absent +``` diff --git a/3 Resources/jq/about.md b/3 Resources/jq/about.md new file mode 100644 index 0000000..09da541 --- /dev/null +++ b/3 Resources/jq/about.md @@ -0,0 +1,11 @@ +--- +tags: + - jq + - json +--- +`jq` is a filter program. Everything , including literals, are `filters` in jq. +filter have input and output. +ops that combine filters like addition or division feed same input to both filters before doing the operation e.g `[1 2 3] | add / length` < array is passd to both add and length and then de division is done. + +identity filter use quotes or brackets and quotes if name has special values such as whitespace. `."Fo o"` || `.["Fo o"]` + diff --git a/3 Resources/jq/arrays.md b/3 Resources/jq/arrays.md new file mode 100644 index 0000000..83373c2 --- /dev/null +++ b/3 Resources/jq/arrays.md @@ -0,0 +1,10 @@ +--- +tags: + - jq +--- +To access a specific index we can just access it by index nr +``` +Command jq '.[0]' +Input [{"name":"JSON", "good":true}, {"name":"XML", "good":false}]| +Output {"name":"JSON", "good":true}| +```
\ No newline at end of file diff --git a/3 Resources/libvirt/Using Virsh to manage bitmaps.md b/3 Resources/libvirt/Using Virsh to manage bitmaps.md new file mode 100644 index 0000000..5cb41f3 --- /dev/null +++ b/3 Resources/libvirt/Using Virsh to manage bitmaps.md @@ -0,0 +1,23 @@ +--- +tags: + - libvirt + - qemu +references: + - https://qemu-project.gitlab.io/qemu/interop/bitmaps.html +--- +To query info about block devices, including bitmaps we can use a QMP command: +``` +sudo virsh qemu-monitor-command --pretty instance-00026133 '{"execute": "query-block", "arguments": {}}' +``` + +There are six primary bitmap-management API commands: +- `block-dirty-bitmap-add` +- `block-dirty-bitmap-remove` +- `block-dirty-bitmap-clear` +- `block-dirty-bitmap-disable` +- `block-dirty-bitmap-enable` +- `block-dirty-bitmap-merge` +And one related query command: +- `query-block` + +The `node` argument to bitmap cmds can be found as `node-name` in `query-block` results.
\ No newline at end of file diff --git a/3 Resources/sysapi/create_cluster.json b/3 Resources/sysapi/create_cluster.json new file mode 100644 index 0000000..e40067e --- /dev/null +++ b/3 Resources/sysapi/create_cluster.json @@ -0,0 +1,9 @@ +{ + "data": { + "attributes": { + "legacy_domain": false, + "region_name": "EU" + }, + "type": "clusters" + } +} diff --git a/3 Resources/sysapi/create_cluster.sh b/3 Resources/sysapi/create_cluster.sh new file mode 100755 index 0000000..d07b48f --- /dev/null +++ b/3 Resources/sysapi/create_cluster.sh @@ -0,0 +1,5 @@ +#!/usr/bin/env nix-shell +#! nix-shell -i bash -p curl + +curl -d @create_cluster.json -n https://systemsapi.one.com/v3/brands/test.groupone.dev/clusters + diff --git a/3 Resources/sysapi/enable_vps.json b/3 Resources/sysapi/enable_vps.json new file mode 100644 index 0000000..1c5567a --- /dev/null +++ b/3 Resources/sysapi/enable_vps.json @@ -0,0 +1,8 @@ +{ + "data": { + "attributes": { + "provisioned": true + }, + "type": "services" + } +} diff --git a/3 Resources/sysapi/patch_vps.sh b/3 Resources/sysapi/patch_vps.sh new file mode 100755 index 0000000..63b181b --- /dev/null +++ b/3 Resources/sysapi/patch_vps.sh @@ -0,0 +1,5 @@ +#!/usr/bin/env nix-shell +#! nix-shell -i bash -p curl + +curl -X PATCH -d @enable_vps.json -n https://systemsapi.one.com/v3/brands/test.groupone.dev/clusters/c0efmoghy/services/vps + |