[{"content":"This will be an about page.\n","permalink":"https://tetragir.com/about/","summary":"\u003cp\u003eThis will be an about page.\u003c/p\u003e","title":"About"},{"content":"I use ZFS on all hard drives. The current setup, is that I have 2 hard drives in no raid or redundancy, just in two VDEVs in a ZFS Pool because I have more data than the capacity of one hard drive. This is then backed up irregularly to another system, where the same situation is set up, 2 VDEVs. That means, I have no redundancy on either of these systems.\nNow with the addition of the new hard drives, I\u0026rsquo;m able to make the primary system at least redundant (and expand the capacity), while expanding the backup capacity too (however, still no redundancy there). As the newly purchased hard drives will go to the backup system as well as I\u0026rsquo;m changing VDEVs, I need to shuffle data around. I have a ZFS pool, called storage on the primary system.\nPlease note, that I\u0026rsquo;m showing the path of all hard drives below, but I actually use WWNs to make sure, that even if the path changes, I use the same hard drives everywhere.\nThe plan is the following:\nCreate a new pool, named storage_tmp with the new hard drives in mirror mode (they are large enough to accommodate all data and I\u0026rsquo;ll gain redundancy immediately) using: sdh sdi Create the snapshot transferon the old pool storage Send the snapshot storage@transferto the pool storage_tmp Once prepared for the change, create a new snapshot transfer_final on storage pool Stop all processes that would access data on storage Send the incremental snapshot storage@transfer_finalto storage_tmp Double check if all data is copied to storage_tmp Destroy the storage pool Create the storage pool with RaidZ2, using sde sdd sda sdf Move back the snapshot storage_tmp@transfer_final to the newly created pool storage Restart applications export the temporary pool storage_tmp Preparation The state of the storage pool, before starting.\n1 2 3 4 5 6 7 8 9 10 11 tetragir@nas:~$ zpool status -L storage pool: storage state: ONLINE config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 sde ONLINE 0 0 0 sdd ONLINE 0 0 0 errors: No known data errors Create Temporary Pool Create the pool storage_tmp\n1 tetragir@nas:~# zpool create -O atime=off storage_tmp mirror /dev/sdh /dev/sdi Confirm that the new pool is created correctly as mirror\n1 2 3 4 5 6 7 8 9 10 11 12 tetragir@nas:~$ zpool status -L storage_tmp pool: storage_tmp state: ONLINE config: NAME STATE READ WRITE CKSUM storage_tmp ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 sdh ONLINE 0 0 0 sdi ONLINE 0 0 0 errors: No known data errors Transfer data from storage to storage_tmp Create the transfer snapshot\n1 tetragir@nas:~# zfs snap -r storage@transfer Start the copy of the storage@stransfer snapshot to storage_tmp\n1 root@nas:~# zfs send -R storage@transfer | zfs recv -Fvu storage_tmp I\u0026rsquo;m using the flags:\nzfs send -R to recursively send all snapshots before this snapshot as well as sending all snapshots recursively zfs recv -F to force a rollback on the destination (this is unnecessary in this case. However, if there would be an older snapshot received on the destination and the data woul\u0026rsquo;ve been changed, this would roll back the dataset) zfs recv -v to show the progress of the transfer zfs recv -u to not mount the filesystem after receiving it Once I\u0026rsquo;m ready to the last data transfer, stop all applications that might use the data. Then create a new snapshot and send an incremental backup with ZFS send.\n1 2 root@nas:~# zfs umount storage root@nas:~# zfs snap -r storage@transfer_last Send the incremental\n1 root@nas:~# zfs send -RI storage@transfer storage@transfer_last | zfs recv -Fvu storage_tmp The flag -I (capital i) defines the 2 snapshots, in between all snapshots will be replicated. In the above command, ZFS will send all snapshots that were created since the creation of the @transfer snapshot until (and including) @transfer_last.\nMake sure, that the received data is correct\n1 root@nas:~# zpool scrub storage_tmp Prepare the new storage pool Double check that you have all data, then destroy the old pool. This is irreversible and will delete ALL data on the old pool! Always triple check!\n1 root@nas:~# zpool destroy storage Create the new pool\n1 zpool create -f -O atime=off -O compression=zstd -o autoexpand=on storage raidz2 /dev/sda /dev/sdd /dev/sde /dev/sdf I use the flag -f, because not all my disks are the same size\nSend the data back from storage_tmp to storage Send back all the data to the storage pool\n1 root@nas:~# zfs send -R storage_tmp@transfer_last | zfs recv -Fv storage Once the transfer is complete, double-check if the is copied back. The filesystems should be mounted, but if not, mount them.\n1 root@nas:~# zfs mount storage Export storage_tmp to make sure, that it\u0026rsquo;s not used anymore.\n1 root@nas:~# zpool export storage_tmp Restart applications and check if they are up and running.\nSummary With this method, I was able to move data temporarily to a different pool, change my primary pool and move data back.\nZFS is awesome.\n","permalink":"https://tetragir.com/posts/2026/01/shuffle-data-to-expand-storage/","summary":"I was able to purchase two new hard drives with a Black Friday offer on a Thursday. I\u0026rsquo;m expanding and also making my storage on my Home Server redundant. this article is mostly for myself to know, what I did.","title":"Shuffle Data to Expand Storage"},{"content":" This post was first published on June 20, 2019 ! Its contents might be outdated!\nSo, I wanted to replace FreeBSD with FreeNAS on my home NAS as well as on my parents’. Both systems are replicating over any new data overnight so in case any of them breaks, we still have our data. I knew I wanted use the built-in solution in FreeNAS which only requires an SSH connection to the other NAS. I was also sure, that even though SSH is safe, I don’t want to replicate data over plain internet. This would also require a fix public IP address on both sides which is not easy with home commercial internet connections. A workaround would be to use dynamic DNS like duckdns or dyndns, but it still requires setting up NAT on the router not to mention exposing your FreeNAS to the public internet (in my experience, most ISP-provided routers do not have a firewall or it’s so dumb that it cannot filter based on source addresses).\nLong story short, I needed VPN. I have a fix IP at home, which makes setup more easy, but this can be also achieved with a cheap VPS running somewhere, acting as a relay.\nRequirements As I mentioned, you need at least one fix IP address somewhere which will be used by the OpenVPN Server. It can be anything, as OpenVPN can run on different platforms. This guide assumes that an OpenVPN Server is already set up.\nThe OpenVPN Server This guide won’t go into details of setting up the OpenVPN Server, but I’ll mention the configuration options that are needed in order to make this connection work.\nThe easiest and safest authentication method for OpenVPN is to use certificates. If a client is trying to connect, the server will check the clients certificate against its CA. If it’s a match, the client is authenticated. The client can/will also check the server’s certificate in order to make sure that it connects to a valid server. There are a lot of guides out there that explain how to set up a PKI (Private Key Infrastructure) with easyrsa (which is installed along with OpenVPN) or other tools.\nThe routes of the client FreeNAS’s network need to be configured in the OpenVPN server. You can use this tool to find out the network and subnet mask: http://www.subnet-calculator.com/ Please note, that the location of the openvpn configuration file could be different. Here I’m using Debian. Once you have it, add it to the server’s configuration:\n1 route 192.168.0.0 255.255.255.0 Besides this, the server needs to know over which client this subnet is reachable. This is achieved with client specific configurations in the Client Config Directory (CCD). If there is none yet, define the CCD location as follows:\n1 client-config-dir /etc/openvpn/ccd Inside this directory, there should be a file corresponding to the client’s name (defined in its certificate). The content of the file is:\n1 2 # cat /etc/openvpn/ccd/client iroute 192.168.0.0 255.255.255.0 You may notice that this is the same subnet defined in the OpenVPN Server configuration file. In the main configuration file, we define that a subnet will be handled on the system by OpenVPN. But because the client will not communicate to the server that this specific subnet is reachable through it, we need to tell the OpenVPN server. This way, once the connection is estbalished, a new route will be added to the operating system’s routing table pointing to the correct client.\nThe client also needs to be informed about the server’s subnet. Add the subnet you want to push to the client:\n1 push \u0026#34;route 10.0.1.0 255.255.255.0\u0026#34; Once the client is connected, it will be informed about this route.\nFreeNAS Settings Creating a Jail for OpenVPN is mostly straightforward, but there are a few gotchas. Create a new Jail by navigating to Jails \u0026gt; Add and continue with Advanced Jail Creation. Assign a name and IP address to the Jail. It is important to check the box to VNET! The Jail has to use VNET in order to make this work. Open Custom Properties and check allow_tun. Besides this, as we will route traffic through the OpenVPN Jail, we need to tell FreeNAS how that server is reachable. Open Network \u0026gt; Static Routes and then Add. Enter the Destination (which is the network we would like to reach this FreeNAS from, the gateway (which is the IP address of the newly created Jail) and a description.\nOpenVPN Setup in the Jail Once the Jail is running, enter it through the CLI (either by SSH into the NAS or using the Shell on the FreeNAS GUI and then iocage console jailname). Install openvpn\n1 pkg install openvpn (or install through ports if you prefer that way). After installation, either copy an OpenVPN configuration file that is already in use or set up a new one. Copy a template:\n1 cp /usr/local/share/examples/openvpn/sample-config-files/client.conf /usr/local/etc/openvpn/ Modify the configuration file as needed (most importantly the remote part and the certificates). You can use the following as a template:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 client #We are a client dev tun #Using TUN instead of TAP proto udp #Using UDP remote your.doma.in 1194 #This can be an IP address and a different port as well resolv-retry infinite #If we are connecting to a domain name, try to resolve it infinitely nobind #No need to bind persist-key #Reserve states between restarts persist-tun user nobody #Drop privilege group nobody ca /path/to/ca.crt #Path to the Certificate Authority cert /path/to/client.crt #Path to the certificate used by this client, signed by the CA key /path/to/client.key #Path to the key belonging to the certificate remote-cert-tls server #Check the server certificate if the certificates were set up for this Please note, that this is just a template, you need to modify it to reflect your actual setup. After all files are in place including the configuration and the certificates, you can try to connect to the server.\n1 # openvpn path/to/the/configuration.conf If the connection is unsuccessful, you will see an explanation in the output.\nTest If the client is successfully connected, check if the routing is done correctly. On the client, the subnet of the server should be there:\n1 2 3 4 5 6 7 8 9 10 11 root@client:~ # netstat -r4 Routing tables Internet: Destination Gateway Flags Netif Expire default 192.168.0.1\tUGS epair0b 10.8.0.0/24 10.8.0.1 UGS tun0 10.8.0.1 link#3 UH tun0 10.8.0.2 link#3 UHS lo0 localhost link#1 UH lo0 10.0.1.0/24\t10.8.0.1 UGS tun0 The route 10.0.1.0/24 was pushed from the server. On the server, check if the client’s subnet is present.\n1 2 3 4 5 6 7 root@server:~ # netstat -r4 Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface default 10.0.1.1 0.0.0.0 UG 0 0 0 eth0 10.0.100.0 0.0.0.0 255.255.255.240 U 0 0 0 eth0 10.8.0.0 0.0.0.0 255.255.255.0 U 0 0 0 tun0 192.168.0.0 10.8.0.2 255.255.255.0 UG 0 0 0 tun0 If both are correct, we should be able to ping the client from the server over the VPN tunnel.\n1 2 3 4 root@server:~# ping -S 10.0.1.2 192.168.0.2 PING 192.168.0.9 (192.168.0.9) 56(84) bytes of data. 64 bytes from 192.168.0.9: icmp_seq=1 ttl=64 time=43.7 ms 64 bytes from 192.168.0.9: icmp_seq=2 ttl=64 time=41.3 ms -S defines the source IP address, in this case this is the local interface of the server. If the route was successfully added on the FreeNAS, ping should be also successful to its local IP address. If you need to reach the FreeNAS from a machine other then the VPN server, you either need to add a static route on the other machine or the same on your default gateway.\nIt’s done This is a fairly easy setup and makes sure that the traffic between you and your FreeNAS installation is secure over an OpenVPN tunnel. It is also possible to use a cheap VPS on DigitalOcean, Vultr or any other provider if you want to reach FreeNAS from a location that does not have a fix IP or is behind CGNAT (Carrier Grade NAT).\n","permalink":"https://tetragir.com/posts/2019/06/running-openvpn-client-in-a-freenas-jail/","summary":"Lately I have had less free time, which has resulted in the fact that this is the first article in over a year. Also I’ve been using mostly CentOS at my job and FreeBSD got out of the focus. But that doesn’t mean that I abandoned FreeBSD, just that solutions that require less time to set up and maintain got my attention.","title":"Running OpenVPN client in a FreeNAS Jail"},{"content":" This post was first published on April 15, 2018 ! Its contents might be outdated!\nFeatures and limitations RIPv1 was introduced in 1988 and RIPv2 in 1993 and it became a standard in 1998. It is easy to use, but has a lot of limitations. The biggest such limitation is that the maximum hop count can only be 15, which means in practice that between the farthest two routers, there can be only 13 other routers. The routing domain can be of course bigger. RIPv2 has the ability to use authentication, however the only method for that is using plain passwords or MD5.\nPreparing FreeBSD In order to send packets between different subnets it is essential to know how one can get to the other subnet. If many subnets have only one common router, this is quite easy, as every local subnet is known and every packet sent to an unknown subnet will be forwarded to another router (aka. to the default gateway). The information about which network is where is stored in the routing table. The currently known subnets can be listed with the netstat command, where -r is telling netstat to give us the routing information, -n is not to resolve the IP addresses and -4 is to do it for IPv4.\n1 2 3 4 5 6 7 8 9 10 root@router1# netstat -r4n Routing tables Internet: Destination Gateway Flags Netif Expire default 192.168.122.1 UGS vtnet0 10.0.1.1 link#3 UH lo1 127.0.0.1 link#2 UH lo0 192.168.122.0/24 link#1 U vtnet0 192.168.122.45 link#1 UHS lo0 Each network has an associated cost in order to figure out which way is the best. In RIP, this cost is calculated from the number of hops, where one hop is one router. A directly connected subnet has a hop count of 0. With RIP, the whole routing table is sent to the neighbor which then processes it and sends the updated routing table to its neighbor, and so on. In the following examples, there are two FreeBSD machines on the same subnet (192.168.122.45/24 and 192.168.122.169/24). Additionally, they have loopback interfaces:\nlo1 on router1 with the IP Address of 10.0.1.1/24 lo2 on router2 with the IP Address of 172.16.2.1/24 Naturally not only loopback interfaces can be included in RIP, but other, physical or VLAN interfaces too. Creating a loopback interface and the make it persistent between reboots:\n1 2 3 4 5 6 Router 1: root@router1# ifconfig lo1 create inet 10.0.1.1/24 root@router1# sysrc cloned_interfaces=lo1 cloned_interfaces: -\u0026gt; lo1 root@router1# sysrc ifconfig_lo1=\u0026#34;inet 10.0.1.1/24\u0026#34; ifconfig_lo1: -\u0026gt; inet 10.0.1.1/24 1 2 3 4 5 6 Router 2: root@router2# ifconfig lo2 create inet 172.16.2.1/24 root@router2# sysrc cloned_interfaces=lo2 cloned_interfaces: -\u0026gt; tap2 root@router2# sysrc ifconfig_lo2=\u0026#34;inet 172.16.2.1/24\u0026#34; ifconfig_lo2: -\u0026gt; inet 172.16.2.1/24 The loopback interfaces on each of the routers exists, though they have no information about the interface on the other machine.\nRouter1:\n1 2 3 4 5 6 7 8 9 10 root@router1# netstat -r4 Routing tables Internet: Destination Gateway Flags Netif Expire default 192.168.122.1 UGS vtnet0 10.0.1.1 link#3 UH lo1 localhost link#2 UH lo0 192.168.122.0/24 link#1 U vtnet0 router1 link#1 UHS lo0 Router 2:\n1 2 3 4 5 6 7 8 9 10 root@router2# netstat -r4 Routing tables Internet: Destination Gateway Flags Netif Expire default 192.168.122.1 UGS vtnet0 localhost link#2 UH lo0 172.16.2.1 link#3 UH lo2 192.168.122.0/24 link#1 U vtnet0 router2 link#1 UHS lo0 Pinging the loopback address on router2 from router1 fails, because router1 has no idea where to send the packets.\n1 2 3 4 5 root@router1# ping -v 172.16.2.1 PING 172.16.2.1 (172.16.2.1): 56 data bytes ^C --- 172.16.2.1 ping statistics --- 7 packets transmitted, 0 packets received, 100.0% packet loss In order to be able to reach the subnets behind the routers, the information about the networks has to be exchanged between the two routers. First, enable routing, this makes sure that if router1 receives a packet on the interface with the IP 192.168.122.45, it won’t drop it, but forward it to the loopback interface. Do this on all of the routers.\n1 2 root@router1# sysctl net.inet.ip.forwarding=1 root@router1# echo net.inet.ip.forwarding=1 \u0026gt;\u0026gt; /etc/sysctl.conf Starting routed. RIP on FreeBSD is handled by routed(8). We start routed with 2 flags:\n-s tells routed to advertise the known routes even if there is only one interface. This is necessary if we want to advertise loopback interfaces. -P ripv2 tells routed to use RIPv2. 1 2 3 4 5 root@router1/2# sysrc routed_enable=YES routed_enable: NO -\u0026gt; YES root@router1/2# sysrc routed_flags=\u0026#34;-s -P ripv2\u0026#34; routed_flags: -q -\u0026gt; -s -P ripv2 root@router1/2# service routed start After around 30 seconds, 10.0.1.1 should be known on router2, and also 172.16.2.1 is known on router1.\n1 2 3 4 5 6 7 8 9 10 11 root@router1# netstat -r4n Routing tables Internet: Destination Gateway Flags Netif Expire default 192.168.122.1 UGS vtnet0 10.0.1.1 link#3 UH lo1 127.0.0.1 link#2 UH lo0 172.16.2.1 192.168.122.169 UGH vtnet0 192.168.122.0/24 link#1 U vtnet0 192.168.122.45 link#1 UHS lo0 1 2 3 4 5 6 7 8 9 10 11 root@router2# netstat -r4n Routing tables Internet: Destination Gateway Flags Netif Expire default 192.168.122.1 UGS vtnet0 10.0.1.1 192.168.122.45 UGH vtnet0 127.0.0.1 link#2 UH lo0 172.16.2.1 link#3 UH lo2 192.168.122.0/24 link#1 U vtnet0 192.168.122.169 link#1 UHS lo0 With that, RIP is up and running on our routers. Even though MD5 is the only option for authentication, it is worth it to turn it on, and shortening the hold time from the default 30 minutes is also useful, but having a lot of options in rc.conf is not convenient. RouteD by default looks for the /etc/gateways file for parameters.\nmd5_passwd=P4ssW0rd|1 tells routed to use P4ssW0rd as the authentication password. RouteD will only accept routes if the incoming routing table uses this password with the key ID 1. There can be more then 1 password, with different key IDs. rdisc_interval=N tells routed how often to send the routing table over. The learned routes will stay in the routing table for 3*N. 1 2 3 # echo “ripv2” \u0026gt;\u0026gt; /etc/gateways # echo “md5_passwd=P4ssW0rd|1” \u0026gt;\u0026gt; /etc/gateways # echo “rdisc_interval=10” \u0026gt;\u0026gt; /etc/gateways Make sure that etc/gateway can be only read by root!\n1 # chmod 600 /etc/gateways Remove the ripv2 parameter from rc.conf\n1 2 # sysrc routed_flags=\u0026#34;-s\u0026#34; routed_flags: -s -P ripv2 -\u0026gt; -s Finally restart routed\n1 2 3 # service routed restart Stopping routed. Starting routed. If the password matches, the route will still be propagated between the routers.\n1 2 3 4 5 6 7 8 9 10 11 root@router1# netstat -r4 Routing tables Internet: Destination Gateway Flags Netif Expire default 192.168.122.1 UGS vtnet0 10.0.1.1 link#3 UH lo1 localhost link#2 UH lo0 172.16.2.1 router2 UGH vtnet0 192.168.122.0/24 link#1 U vtnet0 router1 link#1 UHS lo0 1 2 3 4 5 6 7 8 9 10 11 root@router2:/home/tetragir # netstat -r4 Routing tables Internet: Destination Gateway Flags Netif Expire default 192.168.122.1 UGS vtnet0 10.0.1.1 router1 UGH vtnet0 localhost link#2 UH lo0 172.16.2.1 link#3 UH lo2 192.168.122.0/24 link#1 U vtnet0 router2 link#1 UHS lo0 If the necessary routes are present on both routers, ping will work.\n1 2 3 4 5 6 7 8 9 root@router1:/home/tetragir # ping 172.16.2.1 PING 172.16.2.1 (172.16.2.1): 56 data bytes 64 bytes from 172.16.2.1: icmp_seq=0 ttl=64 time=0.433 ms 64 bytes from 172.16.2.1: icmp_seq=1 ttl=64 time=0.549 ms 64 bytes from 172.16.2.1: icmp_seq=2 ttl=64 time=0.499 ms ^C --- 172.16.2.1 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.433/0.494/0.549/0.048 ms Security and alternatives Using MD5 is not the best to secure RIP, but this is unfortunately the only way (other then plain passwords of course\u0026hellip;). Malicious hosts can advertise routes with better hop count then the other, forcing traffic to flow to them. In bigger networks, OSPF or even BGP can be used to propagate routes between routers.\n","permalink":"https://tetragir.com/posts/2018/04/rip-freebsd/","summary":"I mean Routing Information Protocol of course, not rest in peace! RIP is one of the most basic and easy-to-use routing protocols there is. It has its limitations, but it can be a viable solution for small environments. With RIP it is possible to exchange routing information between devices, and as it is a standard protocol, not only FreeBSD or Linux can use it, but it’s also implemented in many different networking devices, such as routers, firewalls, and even some layer 3 switches.","title":"RIP FreeBSD"},{"content":" This post was first published on January 14, 2018 ! Its contents might be outdated!\nIP Addresses and Netmasks Networking is essential for the operation of a bigger environment. But what is a network? And what do we know about one?\nA network is defined by it\u0026rsquo;s IP Address and netmask and sometimes a VLAN ID. In order to define an IP Address range a netmask is used to say which bits in an IP Address are part of the network identifier and which are part of the host address. An IP Address contains 4 octets. Every IP Address can be translated into a bit stream.\nTake this network for example: 172.16.1.0/26, or 172.16.1.0 with netmask 255.255.255.192 Translate it to binary: 10101100 . 0010000 . 00000001 . 00000000 with netmask 11111111 . 11111111. 11111111. 11000000. Basically the \u0026ldquo;1\u0026rdquo; in the netmask defines which bits in the IP Address are part of the network identifier and where the assignable IP Address space begins.\nThe first 26 bit is a \u0026quot;1\u0026quot; and therefore the first 26 bit in the IP Address is the identifier of the network itself. This means that every IP Address beginning with 10101100 . 0010000 . 00000001 . 00xxxxxx belongs to one network.\nThe last 6 bits define the individual IP Address that are part of this network. In other words the first assignable addresses is: 10101100 . 0010000 . 00000001 . 000000000 and the last assignable address is: 10101100 . 0010000 . 00000001 . 00111111. Translated back to decimal, the first assignable address is: 172.16.1.0 and the last one is 172.16.1.63.\nSubnetting could be calculated on paper, or there are subnet calculator tools out there, like: http://www.subnet-calculator.com/\nThere are 2 \u0026ldquo;special\u0026rdquo; addresses in every network that can never be assigned to any host. These are the first and the last addresses. The first address identifies the network itself, the last one is the broadcast address. If a host sends a packet to the broadcast address, every other host in this particular subnet gets it and processes it.\nTo get back to our example, therefore the assignable addresses in this network are: 172.16.1.[1-62]. It is important to always include the subnet mask with the IP Address because it can cause misunderstandings. For example, one host is configured with the IP Address 172.16.1.100/24 and another with 172.16.1.10/26. As the network identifier and the broadcast addresses are standard and calculated from the IP Address and the subnet mask, the broadcast address will differ on these two hosts.\nThe first one will think, that the broadcast address is 172.16.1.255 while the second one will think it is 172.16.1.63. So if the first host sends a broadcast message to the 172.16.1.255, the second host will never process it, because this packet exists in another subnet from the second hosts point of view. Obviously, this is a problem. It is thus very important that every host in a network be configured with the same subnet mask.\nOf course this does not mean that in one environment only one kind of a netmask can exist! The netmasks can be tailored to the purpose of the network given that the networks are not overlapping! For example, it is perfectly normal to use the following networks in one environment:\n172.16.1.0/26 =\u0026gt; Network ID is 172.16.1.0, broadcast is: 172.16.1.63 172.16.1.64/26 =\u0026gt; Network ID is 172.16.1.64, broadcast is: 172.16.1.127 172.16.1.128/25 =\u0026gt; Network ID is 172.16.1.128, broadcast is: 172.16.1.255 172.16.2.0/24 =\u0026gt; Network ID is 172.16.2.0, broadcast is: 172.16.1.255 IP Address ranges The usable IPv4 Address range is divided and assigned for either private or public use. There are 3 ranges that can be used privately, either at home or in an enterprise environment. These ranges are not routed on the internet! These ranges are:\n10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 These address ranges can be divided into smaller subnets. A subnet is a part of a bigger network, for example the network 172.16.1.0/25 is a subnet of 172.16.1.0/24 because because every IP address in the 172.16.1.0/25 subnet is also part of the 172.16.1.0/24 network. Remember, that every host has to have the same netmask configured! Subnet only means, that a given network is a part of another. If the network 172.16.1.0/24 is divided into 2 subnet, it means that we have the following networks:\n172.16.1.0/25 172.16.1.128/25 In order to send packets between these two subnets, a router is needed with an interface in both subnet.\nThere are more special ranges in IPv4, more about this on the Wikipedia: https://en.wikipedia.org/wiki/IPv4\nVLANs VLAN is Virtual LAN. In a network where more than one subnet exists, VLANs are used to better separate the different networks from each other. A VLAN is basically an identifier assigned to a given subnet by the network administrator. The identifier number is limited by the software both on the computers or on the switches. For example the switch I use supports up to 255 VLANs and the identifier has to be between 1 and 4094. A few VLAN is pre-defined. VLAN 1 is always the default VLAN. If no VLAN is configured on the hosts, that means every host is in VLAN 1. There are a few other VLANs that cannot be used, only for special purpuoses. The VLAN assignment has to be consistent in the whole environment. In the following articles I will use the following networks:\n172.16.1.0/26 =\u0026gt; VLAN 10 172.16.2.0/24 =\u0026gt; VLAN 20 The VLAN identifier could be completely different from the network identifier, it is only useful if from a VLAN identifier the network could be determined without using the documentation.\nOn one single cable, more than one VLAN could be present. In order to achieve that, both interfaces has to be configured to support VLAN tags. If one side sends a packet, the other side has to know which VLAN it belongs to. Therefore if a packet is sent in VLAN 20, the VLAN Identifier will be included in the packet, so the other side knows that the given packet belongs to that speicific VLAN. On one single cable, one untagged VLAN could exist, this called the native VLAN. That means that if both sides are configured with the same native VLAN, both side knows, that if a packet arrives without any VLAN tag, it belongs to that specific VLAN. In the following articles the FreeBSD machine which hosts the virtual machines has an interface both in VLAN 10 and VLAN 20. If native VLANs are in use, be sure to configure the same VLAN on both sides!\nMy network is configured so, that if a packet is being sent in VLAN 10, it is not tagged, but if it belongs to VLAN 20, a VLAN tag is included in the packets.\nThe first 3 layers of the OSI Model the OSI Model describes basically how communication works on a network. There are 8 layers in the OSI Model, I only cover the first 3 as those are needed to understand why we need a router or a switch. I won\u0026rsquo;t cover the OSI model in depth, only so that one can understand what is the difference between a Layer 2 or 3 device. More on Cisco\u0026rsquo;s website: http://docwiki.cisco.com/wiki/Internetworking_Basics Layer 1 is the physical layer. The function of this layer is to define how data is physically transmitted between devices. A common standard used nowadays is the Ethernet Physical Layer. It defines the physical connector used on the cables, the voltage levels, the speed, basically everything physical. This is the IEEE 802.3 standard. Most common form of this standard is a cable with 8P8C connector, or RJ45. This connector is found on PCs, Notebooks, switches, servers, etc. The \u0026ldquo;quality\u0026rdquo; of the cable is also defined. This could be Cat-5, Cat-5e, Cat-6 or other. This defines how the individual cables are shielded (or not). The commonly used optical connections are also ethernet standards. A typical example of a Layer 1 networking device is a hub. Hubs usually has more interfaces. If the bridge receives a bit on one interface, it forwards that to every other interface without looking into the packet. There are no addressing in layer 1. Fortunately hubs died out long ago.\nThere are other standards defining the physical layer of the OSI Model, for example USB, Bluetooth, many IEEE 802.11 standards for Wi-Fi, etc. The second layer called Data Link. This layer is to define, what we communicate if a Layer 1 connection is already negotiated. Ethernet is also used in Layer 2. It defines how a packet looks like, what should it contain, etc. In layer 2, addressing exists, these are the MAC Addresses. By design a MAC Address is unique in the whole world. It is although not entirely true as there are a limited number of MAC Address available. Therefore MAC Addresses are reused by manufacturers on computers that has very little chance happening to be on one network, for example 2 interface from the same manufacturer with identical MAC Addresses could be sold in different continents.\nA typical Layer 2 device is a switch. It can send frames based on the sender and receiver MAC Address. If the receiver is known by the switch, it only sends frames on a specific cable where the receiver is connected. If the receiver is not known, the switch will send out the packet on all of the interfaces belonging to the same VLAN as the sender, and record the MAC into the MAC Address Table only when an answer is received. It is Layer 3 where things are getting interesting. An IP Address for example a Layer 3 address. A switch is not able to route between networks, it operates only in Layer 2 (yes, there are several Layer 3 switches, more on that later). In order to route between the subnets, a router is needed. A router is basically a device that can have more than 1 interface in different subnets. This could either be different physical interfaces for all of the subnets, or VLAN interfaces.\nSwitching and Routing As mentioned before, switches and routers are needed to operate a network. The difference between them is what are the used for.\nA switch is used to share a network between hosts and make it possible that all of the hosts in the same network, or VLAN are able to communicate with each other. From the switches perspective, a router is just a host. When a host is sending packets to a router the switch forwards them to the router. The router then forwards it to the desired network. There are Layer 3 switches which makes networking and routing easy (and cheap). On a layer 3 switch different VLAN Interfaces could coexist and it is able to forward packets between those VLANs. Altough layer 3 switches can route between subnets, a router could do more. Usually a layer 3 switch doesn\u0026rsquo;t know higher level routing protocols, such as OSFP or BGP to begin with.\nConfiguring a Cisco switch to support VLANs In this guide, I use a Cisco switch. It should be similar to configure VLANs on other switches, always refer to the manufacturer\u0026rsquo;s documentation.\nThe Cisco switches CLI interface is accessible from a console interface, Telnet or SSH. Use console for the initial configuration and the switch to SSH. Telnet is not secure, it transmits the passwords, commands unencrypted. Always use SSH. After login, enter global configuration mode:\n1 2 Switch2#configure terminal Switch2(config)# Create the VLAN\n1 Switch2(config)#vlan 10 The VLAN itself is created, it is useful to name it\n1 Switch2(config-vlan)#name Home In order to check if these commands are executed, exit global configuration mode and check the VLAN:\n1 2 Switch2(config-vlan)#end Switch2#show vlan id 10 The output will be similar:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Switch2#show vlan id 10 VLAN Name Status Ports ---- -------------------------------- --------- ------------------------------- 10 Home active VLAN Type SAID MTU Parent RingNo BridgeNo Stp BrdgMode Trans1 Trans2 ---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------ 10 enet 100010 1500 - - - - - 0 0 Remote SPAN VLAN ---------------- Disabled Primary Secondary Type Ports ------- --------- ----------------- ------------------------------------------ Create a second VLAN:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Switch2#conf t Enter configuration commands, one per line. End with CNTL/Z. Switch2(config)#vlan 20 Switch2(config-vlan)#name Guest Switch2(config-vlan)#end Switch2#sh vlan id 20 VLAN Name Status Ports ---- -------------------------------- --------- ------------------------------- 20 Guest active VLAN Type SAID MTU Parent RingNo BridgeNo Stp BrdgMode Trans1 Trans2 ---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------ 20 enet 100020 1500 - - - - - 0 0 Remote SPAN VLAN ---------------- Disabled Primary Secondary Type Ports ------- --------- ----------------- ------------------------------------------ Assign the the 2 VLANs created before to a phisycal interface. VLAN 10 will be the native VLAN. I define the VLANs which are alloved on the wire, this way if the switch gets a packet other then VLAN 10 or 20, the packet will be dropped. Tailer the commands to the interfaces accordingly, I use FastEthernet0/2 on the switch:\n1 2 3 4 5 6 Switch2#conf t Switch2(config)#interface FastEthernet0/2 Switch2(config-if)#switchport trunk native vlan 10 Switch2(config-if)#switchport trunk allowed vlan 10,20 Switch2(config-if)#switchport mode trunk Switch2(config-if)#end Check if the commands are executed:\n1 2 3 4 5 6 7 8 9 10 11 12 13 Switch2#show interfaces FastEthernet 0/2 trunk Port Mode Encapsulation Status Native vlan Fa0/2 on 802.1q trunking 10 Port Vlans allowed on trunk Fa0/2 10,20 Port Vlans allowed and active in management domain Fa0/2 10,20 Port Vlans in spanning tree forwarding state and not pruned Fa0/2 10,20 If the result is satisfactory, save the configuration\n1 2 3 4 5 Switch2#copy running-config startup-config Destination filename [startup-config]? Building configuration... [OK] Switch2# Next step is to configure the FreeBSD machine connected to the port configured above.\nConfiguring an interface on FreeBSD to support VLANs On my machine, the physical interface is em0, all the commands are referenced to em0. Check the physical interface with ifconfig.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 % ifconfig em0: flags=8843\u0026lt;UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST\u0026gt; metric 0 mtu 1500 options=4219b\u0026lt;RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO\u0026gt; ether 18:03:73:c7:17:14 inet 172.16.1.11 netmask 0xffffffc0 broadcast 172.16.1.63 nd6 options=29\u0026lt;PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL\u0026gt; media: Ethernet autoselect (100baseTX \u0026lt;full-duplex\u0026gt;) status: active lo0: flags=8049\u0026lt;UP,LOOPBACK,RUNNING,MULTICAST\u0026gt; metric 0 mtu 16384 options=600003\u0026lt;RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6\u0026gt; inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21\u0026lt;PERFORMNUD,AUTO_LINKLOCAL\u0026gt; groups: lo If the physical interface differs, tailer the commands accordingly! em0 has already got an IP address. As VLAN 10 is the native VLAN to this machine, without any further configuration, this machine will operate in VLAN 10 without knowing it. Creating VLAN 20 and adding an IP Address to it:\n1 # ifconfig em0.20 create vlan 20 vlandev em0 inet 172.16.2.10/24 This command will create and interface, em0.20, assign it to the physical interface, em0 and use tagging. Check the results with ifconfig:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 # ifconfig em0: flags=8843\u0026lt;UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST\u0026gt; metric 0 mtu 1500 options=4219b\u0026lt;RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO\u0026gt; ether 18:03:73:c7:17:14 inet 172.16.1.11 netmask 0xffffffc0 broadcast 172.16.1.63 nd6 options=29\u0026lt;PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL\u0026gt; media: Ethernet autoselect (100baseTX \u0026lt;full-duplex\u0026gt;) status: active lo0: flags=8049\u0026lt;UP,LOOPBACK,RUNNING,MULTICAST\u0026gt; metric 0 mtu 16384 options=600003\u0026lt;RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6\u0026gt; inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 nd6 options=21\u0026lt;PERFORMNUD,AUTO_LINKLOCAL\u0026gt; groups: lo em0.20: flags=8843\u0026lt;UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST\u0026gt; metric 0 mtu 1500 options=103\u0026lt;RXCSUM,TXCSUM,TSO4\u0026gt; ether 18:03:73:c7:17:14 inet 172.16.2.10 netmask 0xffffff00 broadcast 172.16.2.255 inet6 fe80::1a03:73ff:fec7:1714%em0.20 prefixlen 64 scopeid 0x3 nd6 options=21\u0026lt;PERFORMNUD,AUTO_LINKLOCAL\u0026gt; media: Ethernet autoselect (100baseTX \u0026lt;full-duplex\u0026gt;) status: active vlan: 20 vlanpcp: 0 parent interface: em0 groups: vlan In order to preserve the VLAN between reboots, add the following lines to /etc/rc.conf:\n1 2 vlans_em0=\u0026#34;20\u0026#34; ifconfig_em0_20=\u0026#34;inet 172.16.2.10/24\u0026#34; If another host exists in VLAN 20, use ping to prove it works:\n1 2 3 % ping 172.16.2.1 PING 172.16.2.1 (172.16.2.1): 56 data bytes 64 bytes from 172.16.2.1: icmp_seq=0 ttl=64 time=0.425 ms Specify the source interface to be sure the packets are not sourced from VLAN 10 and then routed by a router:\n1 2 3 % ping -S 172.16.2.10 172.16.2.1 PING 172.16.2.1 (172.16.2.1) from 172.16.2.10: 56 data bytes 64 bytes from 172.16.2.1: icmp_seq=0 ttl=64 time=0.455 ms The first ping is okay to be lost, but after that there should be an answer for every ping request.\n","permalink":"https://tetragir.com/posts/2018/01/basic-networking-on-freebsd/","summary":"In this article I would like to introduce the very basics of networking in order to  allow the reader to be able to separate VMs from each other or to organize them into logically different segments. This is far from a detailed description of networks, of course!","title":"Basic Networking on FreeBSD"},{"content":" This post was first published on January 11, 2018 ! Its contents might be outdated!\nNotes This guide is written on, but not limited to FreeBSD. The commands should work on every other system where zfsd is present and should behave the same. I’m writing this guide and take the output from a FreeBSD 11.1 BETA-1 machine with virtual disks.\nThe Spare Usually a zpool contains identical disks. ZFS wont stop you from adding a smaller disk as a spare to a zpool, though it won’t be able to replace any disk with it. It is therefore highly recommended to add a disk which is identical, or has a greater capacity as the other disks in the zpool.\nAdding disks as spares One spare can be a part of more then one zpool. Once a disk fails in a zpool, it will be replaced with the spare. Understandably in this case, if we had only one spare, if one more disk fails, it cannot be replaced automatically. It is possible however to define more than one spare. After consideration, the spare disks can be added to an already existing pool. In the following example, we add two spare disks to two different already existing zpools. We start with 2 datasets, each with 2 disks in a mirror\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 # zpool status tank1 pool: tank1 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank1 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 vtbd3 ONLINE 0 0 0 vtbd4 ONLINE 0 0 0 errors: No known data errors # zpool status tank2 pool: tank2 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank2 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 vtbd5 ONLINE 0 0 0 vtbd6 ONLINE 0 0 0 errors: No known data errors Adding two new disks as spares to both of the zpools:\n1 2 # zpool add tank1 spare vtbd1 vtbd2 # zpool add tank2 spare vtbd1 vtbd2 Both of the spares are showing up in the zpools.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 # zpool status tank1 pool: tank1 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank1 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 vtbd3 ONLINE 0 0 0 vtbd4 ONLINE 0 0 0 spares vtbd1 AVAIL vtbd2 AVAIL errors: No known data errors root@zfsd:/home/tetragir # zpool status tank2 pool: tank2 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank2 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 vtbd5 ONLINE 0 0 0 vtbd6 ONLINE 0 0 0 spares vtbd1 AVAIL vtbd2 AVAIL errors: No known data errors Remove spares from datasets It is also possible to remove the spares from the zpools, even if the pools are online. Make vtbd1 exclusive for tank1 and vtbd2 exclusive for tank2:\n1 2 # zpool remove tank1 vtbd2 # zpool remove tank2 vtbd1 This requires the drives not to be in use by any of the zpools.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 # zpool status tank1 pool: tank1 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank1 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 vtbd3 ONLINE 0 0 0 vtbd4 ONLINE 0 0 0 spares vtbd1 AVAIL errors: No known data errors # zpool status tank2 pool: tank2 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank2 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 vtbd5 ONLINE 0 0 0 vtbd6 ONLINE 0 0 0 spares vtbd2 AVAIL errors: No known data errors Zfsd only cares about the zfs partition, so it will only replace the failed disk, or the zfs partition on the disk. If there were other partitions on the disk, those won’t be affected. For example if a partition is used on the disk as a root on zfs, the bootloader on the other partitions won’t be replicated to the spare. It is of course possible to put the bootloader on the spare at any time. But be careful! If that disk or partition is part of another zpool as a spare, the partition has to be at least as big as the other disks or partitions! The replacement In order to take any action, zfsd has to run. To start it on boot:\n1 2 # sysrc zfsd_enable=YES zfsd_enable: NO -\u0026gt; YES Start it right away:\n1 # service zfsd start Making sure it works These two actions, adding a spare to a zpool and making sure that zfsd runs, are enough to initiate a replacement upon a disk failure, or because of other reasons. In the following examples, we have a pool named tank with two mirrors and two spare disks:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # zpool create tank mirror vtbd3 vtbd4 mirror vtbd5 vtbd6 spare vtbd1 vtbd2 # zpool status tank pool: tank state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 vtbd3 ONLINE 0 0 0 vtbd4 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 vtbd5 ONLINE 0 0 0 vtbd6 ONLINE 0 0 0 spares vtbd1 AVAIL vtbd2 AVAIL errors: No known data errors Vtbd6 has failed and zfsd replaced it with a spare:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 # zpool status tank pool: tank state: DEGRADED status: One or more devices could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Attach the missing device and online it using \u0026#39;zpool online\u0026#39;. see: http://illumos.org/msg/ZFS-8000-2Q scan: resilvered 51.5K in 0h0m with 0 errors on Sun Jun 18 11:38:34 2017 config: NAME STATE READ WRITE CKSUM tank DEGRADED 0 0 0 mirror-0 ONLINE 0 0 0 vtbd3 ONLINE 0 0 0 vtbd4 ONLINE 0 0 0 mirror-1 DEGRADED 0 0 0 vtbd5 ONLINE 0 0 0 spare-1 UNAVAIL 0 0 0 9567955613261044063 UNAVAIL 0 0 0 was /dev/vtbd6 vtbd1 ONLINE 0 0 0 spares 11651124579463483589 INUSE was /dev/vtbd1 vtbd2 AVAIL errors: No known data errors The pool is in DEGRADED mode, showing that a disk is missing, though currently the mirror is still redundant, thanks to the spare that replaced the failed disk. Replacing the failed disk If the autoreplace property is off (# zpool get autoreplace tank), - which is the default property -, and if a new device appears on the same interface, the replacement of the failed disk in zpool has to be initiated manually.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 # zpool replace tank 9567955613261044063 vtbd6 The disk is then resilvered and vtbd1 becomes available again. # zpool status tank pool: tank state: ONLINE scan: resilvered 63.5K in 0h0m with 0 errors on Sun Jun 18 12:23:02 2017 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 vtbd3 ONLINE 0 0 0 vtbd4 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 vtbd5 ONLINE 0 0 0 vtbd6 ONLINE 0 0 0 spares vtbd1 AVAIL vtbd2 AVAIL errors: No known data errors In the case of autoreplace=on, the replacement disk will be automatically used, eliminating the need of the zpool replace step. Setting the property:\n1 2 3 4 # zpool set autoreplace=on tank # zpool get autoreplace tank NAME PROPERTY VALUE SOURCE tank autoreplace on local When a spare is activated zfsd will activate a spare in any of the following cases:\nDevice failure: if a device is removed (failed, or just disconnected) I/O Errors: If a device generates more than 50 I/O Errors in 60 seconds Checksum errors: If a device generates more than 50 checksum errors in 60 seconds ","permalink":"https://tetragir.com/posts/2018/01/zfs-hot-spare-with-zfsd/","summary":"Since FreeBSD 11.0 was released, it\u0026rsquo;s been possible to use a disk as a hot spare in a ZFS zpool. This means, that under a few circumstances, zfsd will take a predefined disk to replace another one.","title":"ZFS Hot Spare with ZFSd"},{"content":" This post was first published on June 3, 2017 ! Its contents might be outdated!\nI remembered that Civilization V is available through Steam, so I checked and it turned out, that it runs on Linux! Currently we use Arch Linux on the main desktop PC: it\u0026rsquo;s an i7 rig with an Nvidia graphics card. Would it work, my wife looked at me hopefully. I could only rise to the challenge.\nPlease note that this short guide is written on June 3rd in 2017 and it is accurate now. Over time, things change and this guide will be outdated. Also this guide is tailored to Arch Linux with a discrete Nvidia graphics card. Though the solution should work on other Linux systems, always consider the differences between distros!\nInstalling Steam The first step is to install Steam. If you run a 32bit system, you can just install it; on 64bit systems the multilib repo has to be enabled. To do that, uncomment the following lines in /etc/pacman.conf\n1 2 [multilib] Include = /etc/pacman.d/mirrorlist Then just install the steam package: pacman -S steam\nFor me, Steam did not run immediately, I also had to install the steam-native-runtime package: pacman -S steam-native-runtime\nRunning on 64bit it is also important to install the 32bit version of the graphics driver. Install the appropriate one according to this page: https://wiki.archlinux.org/index.php/Xorg#Driver_installation In my case it was the Nvidia driver: pacman -S lib32-nvidia-utils\nAfter installing all of these, I was able to start Steam itself.\nInstalling Civilization V Though it didn\u0026rsquo;t seem tricky at first, when I installed the game on an XFS file system it always complained that a file was not found, and after the intro video the game failed to start. (My wife was sad at this point.) The error message was: Unable to load texture (LoadingBaseGame.dds)\nI found just one similar report and it was on macOS, but it made sense so I gave it a try. Someone installed a game on an HFS+ disk with case-sensitivity enabled. I threw in an old 250GB HDD to the machine and formatted it to FAT32. After installing the game to the new disk, it found the file and I was able to start the game.\nSo the solution is to use a file system without case sensitivity.\nStarting the game From here there was only one problem to be solved. According to a few forum topics the new Nvidia driver changed a few things rendering the game unable to start. I got the following error in the terminal upon starting the game:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 GameAction [AppID 8930, ActionID 10] : LaunchApp changed task to Starting with \u0026#34;\u0026#34; GameAction [AppID 8930, ActionID 10] : LaunchApp changed task to SynchronizingCloud with \u0026#34;\u0026#34; GameAction [AppID 8930, ActionID 10] : LaunchApp changed task to CreatingProcess with \u0026#34;\u0026#34; GameAction [AppID 8930, ActionID 10] : LaunchApp waiting for user response to CreatingProcess \u0026#34;\u0026#34; GameAction[AppID 8930, ActionID 10] : LaunchApp continues with user response \u0026#34;CreatingProcess\u0026#34; Game update: AppID 8930 \u0026#34;Sid Meier\u0026#39;s Civilization V\u0026#34;, ProcID 29664, IP 0.0.0.0:0 \u0026gt;\u0026gt;\u0026gt; Adding process 29664 for game ID 8930 GameAction [AppID 8930, ActionID 10] : LaunchApp changed task to WaitingGameWindow with \u0026#34;\u0026#34; ERROR: ld.so: object \u0026#39;/home/user/.local/share/Steam/ubuntu12_32/gameoverlayrenderer.so\u0026#39; from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored. ERROR: ld.so: object \u0026#39;/home/user/.local/share/Steam/ubuntu12_64/gameoverlayrenderer.so\u0026#39; from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored. /mnt/397D-352A/Games/steamapps/common/Sid Meier\u0026#39;s Civilization V/./Civ5XP: Symbol `_ZTVN10__cxxabiv120__si_class_type_infoE\u0026#39; has different size in shared object, consider re-linking /mnt/397D-352A/Games/steamapps/common/Sid Meier\u0026#39;s Civilization V/./Civ5XP: Symbol `_ZTVN10__cxxabiv117__class_type_infoE\u0026#39; has different size in shared object, consider re-linking /mnt/397D-352A/Games/steamapps/common/Sid Meier\u0026#39;s Civilization V/./Civ5XP: Symbol `_ZTVN10__cxxabiv121__vmi_class_type_infoE\u0026#39; has different size in shared object, consider re-linking GameAction [AppID 8930, ActionID 10] : LaunchApp changed task to Completed with \u0026#34;\u0026#34; \u0026gt;\u0026gt;\u0026gt; Adding process 29665 for game ID 8930 Game removed: AppID 8930 \u0026#34;Sid Meier\u0026#39;s Civilization V\u0026#34;, ProcID 29664 No cached sticky mapping in ActivateActionSet. Fortunately this can be solved with a startup parameter. In Steam, in the list of the games, right click on the game and choose properties.\nUnder launch options, paste the following: LD_PRELOAD=/usr/lib/nvidia/libGL.so %command%\nAfter that, the game did start up and did not crash which is good. Since then I lost access to the PC and I hadn\u0026rsquo;t been able to regain it from my wife since. (But she is happy now.)\n","permalink":"https://tetragir.com/posts/2017/06/installing-civilization-v-on-arch-linux/","summary":"I\u0026rsquo;m not really a gamer myself, but as it turned out, my wife is. She played with Civilization V many years ago on Windows, and wanted to re-play the game now. However, I have successfully eliminated it from everywhere in the household, we run Linux on the desktops and FreeBSD on the “servers”, so we seemed to have a slight problem: how to let the wife play without installing anything Windows?","title":"Installing Civilization V On Arch Linux"},{"content":" This post was first published on February 25, 2017 ! Its contents might be outdated!\nIt is similar to Cisco\u0026rsquo;s CDP, Foundry\u0026rsquo;s FDP, Nortel\u0026rsquo;s SONMP, etc. It is a stateless protocol, meaning that an LLDP-enabled device sends advertisements even if the other side cannot do anything with it. In this guide the installation and configuration of the LLDP daemon on FreeBSD as well as on a Cisco switch will be introduced.\nThe LLDP Protocol If you are already familiar with Cisco\u0026rsquo;s CDP, LLDP won\u0026rsquo;t surprise you. It is built for the same purpose: to exchange device information between peers on a network. While CDP is a proprietary solution and can be used only on Cisco devices, LLDP is a standard: IEEE 802.3AB. Therefore it is implemented on many types of devices, such as switches, routers, various desktop operating systems, etc. LLDP helps a great deal in mapping the network topology, without spending hours in cabling cabinets to figure out which device is connected with which switchport. If LLDP is running on both the networking device and the server, it can show which port is connected where. Besides physical interfaces, LLDP can be used to exchange a lot more information, such as IP Address, hostname, etc.\nInstalling lldpd on FreeBSD In order to use LLDP on FreeBSD, net-mgmt/lldpd has to be installed. It can be installed from ports using portmaster:\n1 portmaster net-mgmt/lldpd Or from packages:\n1 pkg install net-mgmt/lldpd By default lldpd sends and receives all the information it can gather , so it is advisable to limit what we will communicate with the neighboring device.\nIf the machine has for example a lot of tap or bridge interfaces, lldpd will advertise all of them to the neighboring switch. Although this can be useful, it makes the output harder to read. It also sends and receives LLDP advertisements on all interfaces, including taps, loopback, wireless, etc. lldpd can be told on which interfaces to run on, which interfaces to advertise, etc. The configuration file for lldpd is basically a list of commands as it is passed to lldpcli. Create a file named lldpd.conf under /usr/local/etc/ The following configuration gives an example of how lldpd can be configured. For a full list of options, see %man lldpcli\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Hostname configure system hostname \u0026#34;Voyager\u0026#34; # Operating System configure system description \u0026#34;FreeBSD 11.0-RELEASE\u0026#34; # Interfaces to send LLDP advertisements on configure system interface pattern em*,!bridge*,!tap* # Management IP Address configure system ip management pattern 10.0.42.120,!*:* # Enable LLDP-MED configure med fast-start enable # Physical location configure med location address country DE city \u0026#34;Frankfurt\u0026#34; street \u0026#34;Some Street\u0026#34; number \u0026#34;Number\u0026#34; name \u0026#34;Some Datacenter\u0026#34; To check what is configured locally, run #lldpcli show chassis detail\nTo see the neighbors run #lldpcli show neighbors details\nIn order to start lldpd on startup, run #sysrc lldpd_enable=YES\nEnabling LLDP on a Cisco Switch Cisco devices usually run CDP by default with LLDP turned off. CDP and LLDP can coexist, but if CDP is not needed, turn it off: Switch(config)#no cdp run Turn on LLDP: Switch(config)#lldp run Checking the devices connected to the switches: Switch#show lldp neighbors\nUnder the first column, Device ID, the hostname of the device is shown without the domain. The next column is called Local Intf. It shows to which local switchport is the device connected to. Hold-Time shows how long the switch holds the information about the connected device. By default the devices are sending advertisements every 30 seconds. If the switch is not getting any new advertisements in the configured hold time, it will delete the neighbor from the list. The Capability column shows what exactly are the connected devices capable of. This value is configurable on the host if we want to hide or specifically show something. The last column is Port ID. This column displays how the neighbor calls the interface. To list the configured LLDP parameters on the switches enter the following command: #show lldp\nA lot more is advertised over LLDP, these details can be seen with the #show lldp neighbors detail command.\n","permalink":"https://tetragir.com/posts/2017/02/using-lldp-on-freebsd/","summary":"LLDP, or Link Layer Discovery Protocol allows system administrators to easily map the network, eliminating the need to physically run the cables in a rack. LLDP is a protocol used to send and receive information about a neighboring device connected directly to a networking interface.","title":"Using LLDP on FreeBSD"},{"content":" This post was first published on May 27, 2016 ! Its contents might be outdated!\nAltough the Cisco Virtual Wireless LAN Controller (vWLC) is officially supported only on VMWare ESXi, Linux KVM and Microsoft Hyper-V, it is relatively easy to install and operate on FreeBSD bhyve. The installation is actually pretty similar to KVM.\nSetup I use the following setup in this guide:\nFreeBSD 11-CURRENT r297234 Cisco Virtual Wireless LAN Controller image 8.2.100.0 (Small Scale Image) Preparation, requirements bhyve and GRUB In order to run any VMs with bhyve, the vmm module has to be loaded in (or compiled in the kernel). To load it in:\n1 # kldload vmm The following is needed for the vmm module to load automatically at startup. It appends a new line to the /boot/loader.conf file.\n1 # echo \u0026#39;vmm_load=\u0026#34;YES\u0026#34;\u0026#39; \u0026gt;\u0026gt; /boot/loader.conf The sysutils/grub2-bhyve also has to be installed either from ports with portmaster,\n1 # portmaster sysutils/grub2-bhyve or from packages.\n1 # pkg install grub2-bhyve The Cisco Virtual Wireless LAN Controller The requirements of the Cisco vWLC are pretty straightforward. With the Small Scale Image, the restrictions are 200 Access Points and 6000 clients maximum, while with the Large Scale Image it is possible to register up to 3000 access points and have up to 32000 clients. In this scenario I use the Small Scale Image.\nThe requirements are (Small Scale / Large Scale):\nCPU: 1 vCPU / 2 vCPU RAM: 2GB / 8GB Disk space: 8GB / 8GB Network Interfaces: 2 vNIC / 2vNIC The only difference is with the VMs and install images; the licensing and the controllers are the same. More can be read about the Cisco Virtual Wireless LAN Controller on Cisco\u0026rsquo;s website.\nPrepare the environment In order to install the vWLC, the installer ISO is needed: this can be downloaded from Cisco\u0026rsquo;s website.\nNow create the file used as storage.\n1 # truncate -s 8G vwlc.img Create a device.map file for GRUB identifying which file is which device. The contents are:\n1 2 (hd0) ./vwlc.img (cd0) ./MFG_CTVM_8_2_100_0.iso (The following files are in one folder)\n1 MFG_CTVM_8_2_100_0.iso\tdevice.map\tvwlc.img Install The two network interfaces are not needed here, and thus they will be created later on.\nThe installation process of the Cisco vWLC is pretty easy, since the ISO file contains an automated installer. Once the installer ISO is booted up, it looks for the virtual hard drive at /dev/sda. If it finds it and the disk is empty, a script installs the vWLC software and then reboots. Everything is automatic, the only interaction actually needed is in the case of a previous installer present on the virtual hard drive.\nActually loading the installer ISO is not so straightforward however, as Cisco uses GRUB version 1. Fortunately grub2-bhyve can load the legacy config file.\nStart GRUB and point it to the legacy config file:\n1 # grub-bhyve -m device.map -r cd0 -M 2048M vwlc 1 2 3 grub\u0026gt; ls (cd0)/boot/grub/ menu.lst menu_vm.lst stage2_eltorito grub\u0026gt; legacy_configfile (cd0)/boot/grub/menu.lst After that, the virtual machine can be started. Right now only the console, the hard disk (with ahci-hd emulation!) and the installer ISO is needed. Besides that, 1vCPU and 2048MB of RAM is defined.\n1 2 3 4 5 6 7 8 9 bhyve -A -H -P \\ -s 0:0,hostbridge \\ -s 1:0,lpc \\ -s 2:0,ahci-hd,./vwlc.img \\ -s 3:0,ahci-cd,./MFG_CTVM_8_2_100_0.iso \\ -l com1,stdio \\ -c 1 \\ -m 2048M \\ vwlc The installation is automatic and takes little time. After a successful install, the Virtual Machine powers down.\nNote: It is important to \u0026ldquo;destroy\u0026rdquo; the VM! To do this, first check which VMs are running:\n1 2 # ls /dev/vmm vwlc Then stop the vWLC.\n1 # bhyvectl --vm=vwlc --destroy First start, setup From this point, the Cisco vWLC needs two vNIC in order to operate, else it will crash during startup. The next few lines define two tap interfaces and a bridge between the first one and the physical interface. The name of the physical interface can differ, check with ifconfig!\n1 2 3 4 5 6 7 # ifconfig tap0 create # ifconfig tap1 create # sysctl net.link.tap.up_on_open=1 net.link.tap.up_on_open: 0 -\u0026gt; 1 # ifconfig bridge0 create # ifconfig bridge0 addm *em0* addm tap0 # ifconfig bridge0 up Load GRUB and point it to the legacy config file\n1 # grub-bhyve -m device.map -r hd0,msdos1 -M 2048 vwlc 1 grub\u0026gt; legacy_configfile /grub/menu.lst After that the VM could be started similarly as before, but now without the installer ISO and with the two newly created vNICs.\n1 2 3 4 5 6 7 8 9 10 # bhyve -A -H -P \\ -s 0:0,hostbridge \\ -s 1:0,lpc \\ -s 2:0,ahci-hd,./vwlc.img \\ -s 3:0,virtio-net,tap0 \\ -s 4:0,virtio-net,tap1 \\ -l com1,stdio \\ -c 1 \\ -m 2048M \\ vwlc The setup of the vWLC is now loaded and it can be configured. After the setup process, it reboots.\nNote: It is important to destroy the instance after every shutdown!\n1 # bhyvectl --vm=vwlc --destroy Operation While starting the vWLC with the bhyve command above, the current console session is lost. Therefore it is advisable to use a nulldevice for that very reason. The GRUB process is the same as above, the difference is only defining the console device for the virtual machine.\n1 2 3 4 5 6 7 8 9 10 bhyve -A -H -P \\ -s 0:0,hostbridge \\ -s 1:0,lpc \\ -s 2:0,ahci-hd,./vwlc.img \\ -s 3:0,virtio-net,tap0 \\ -s 4:0,virtio-net,tap1 \\ -l com1,/dev/nmdm0A \\ -c 1 \\ -m 2048M \\ vwlc \u0026amp; Connect to the nulldevice\u0026rsquo;s other end:\n1 cu -l /dev/nmdm0B If the vWLC was properly set up, it is now available to log in from a web browser.\nSummary Unfortunately, running the vWLC on FreeBSD bhyve is not yet officially supported by Cisco, but it boots up and looks like it\u0026rsquo;s working. By default the vWLC comes with a 90 days evaluation license for up to 200 AP. After the 90 days the vWLC will continue to work but will deny any APs trying to register.\n","permalink":"https://tetragir.com/posts/2016/05/cisco-virtual-wireless-controller-on-freebsd-bhyve/","summary":"In this guide a Cisco vWLC will be installed on FreeBSD bhyve.","title":"Cisco Virtual Wireless Controller on FreeBSD Bhyve"},{"content":" This post was first published on March 27, 2016 ! Its contents might be outdated!\nI\u0026rsquo;ve read a nice tutorial about letsencrypt and nginx on Peter Wemm\u0026rsquo;s site. This guide is similar, but for an h2o webserver. I intend to write another guide about www/h2o later once v2.0 is released, this is just a short tutorial about letsencrypt.'\nRequirements You only need the www/h2o webserver (obviously) and the security/letsencrypt.sh.\nPreparation First, install security/letsencrypt.sh from ports (with portmaster):\n1 portmaster security/letsencrypth.sh Or from packages:\n1 pkg install security/letsencrypt.sh Then configure h2o to redirect the domain validation request to the right folder.\n1 2 3 paths: \u0026#34;/.well-known/acme-challenge\u0026#34;: file.dir: \u0026#34;/usr/local/etc/letsencrypt.sh/.acme-challenges\u0026#34; The above needs to be in the part where h2o is configured to listen on port 80. The other important thing is to place this before the redirection to https (if any). Then restart h2o.\n1 service h2o restart Generate the certificates Create a config.sh file containing the contact email address, and a domains.txt file with the domain (both with and without \u0026ldquo;www.\u0026rdquo;) and request a certificate. Replace tetragir.com with the actual domain. The options that can be configured are explained in the /usr/local/etc/letsencrypt.sh/config.sh.example file.\n1 2 3 4 cd /usr/local/etc/letsencrypt.sh echo \u0026#39;CONTACT_EMAIL=your@email.address\u0026#39; \u0026gt; config.sh echo \u0026#39;tetragir.com www.tetragir.com\u0026#39; \u0026gt; domains.txt letsencrypt.sh Once you are done, the result should be the following:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # INFO: Using main config file /usr/local/etc/letsencrypt.sh/config.sh Processing tetragir.com + Signing domains... + Generating private key... + Generating signing request... + Requesting challenge for tetragir.com... + Responding to challenge for tetragir.com... + Challenge is valid! + Requesting certificate... + Checking certificate... + Done! + Creating fullchain.pem... + Done! If everything went right, the certificate can be found in the /usr/local/etc/letsencrypt.sh/certs/tetragir.com/ folder.\nConfigure h2o to the newly created certificates H2O needs to be told where these certificates actually are, so the following lines need to be placed in the part where h2o is configured to listen on port 443.\n1 2 certificate-file: /usr/local/etc/letsencrypt.sh/certs/tetragir.com/fullchain.pem key-file: /usr/local/etc/letsencrypt.sh/certs/tetragir.com/privkey.pem Naturally you need to replace the folder names with the actual path. Then restart h2o.\n1 service h2o restart Automating letsencrypt The certificates from letsencrypt are only valid for 90 days and therefore it is advisable to automate the process. It is possible to run letsencrypt.sh with cron, but I like the \u0026ldquo;periodic\u0026rdquo; solution more. In order to make it work, place the following line in /etc/periodic.conf:\n1 weekly_letsencrypt_enable=\u0026#34;YES\u0026#34; This way the server will check for certificate renewals every week and will renew it when necessary (that is, when remaining validity time is shorter than 30 days). Have fun!\n","permalink":"https://tetragir.com/posts/2016/03/using-letsencrypt.sh-with-h2o/","summary":"This will be a really short guide about how to set up the H2O webserver with letsencrypt, and how to automate it.","title":"Using letsencrypt.sh with h2o"},{"content":" This post was first published on January 1, 2016 ! Its contents might be outdated!\nThe guide is no longer accurate mainly because FreeBSD 10.3 came out and also 11-CURRENT doesn\u0026rsquo;t need to be modified in order to boot on the Zero. You can follow this guide with the 11-CURRENT image, it will work without modifying the files on the boot partition. The images could be downloaded from the FreeBSD ARM images page. Look for the RPI-B image. The downloaded image can easy copied to the microSD card and it will boot. There is no need to modify the files in the boot partition.\nScope of this guide This guide gives instructions on how can one boot FreeBSD 10.2-STABLE on a Raspberry Pi Zero. As a result, the operating system boots up, and is available over SSH. The instructions for the activities on the SD card are written and done on a Fedora 23, but they should apply to many other Linux/UNIX systems. I use the following additional components:\nThe SD card: Samsung microSD EVO+ 32GB (MB-MC32DAAMZ)\nThe microUSB - Ethernet adapter: Icy Box IB-AC510\nPreparing the SD card On my system, the SD card came up as /dev/sdb. It is possible that on other systems it would be something else. Checking the dmesg or lsblk could be very helpful. The first step is to “format” the SD card. I zero out the card before doing anything on it.\n1 2 3 4 5 # dd if=/dev/zero of=/dev/sdb bs=4M dd: error writing ‘/dev/sdb’: No space left on device 7633+0 records in 7632+0 records out 32010928128 bytes (32 GB) copied, 2205.6 s, 14.5 MB/s This process usually takes a long time, but depends on the size and maximal speed of the SD card.\nDownloading and writing the FreeBSD image to the SD card In the meantime, download the actual FreeBSD image and unxz it. I use the 10.2-STABLE image, but it should work with the 11-CURRENT image too. The up-to-date images are available on the FreeBSD ARM Download page. After downloading, the file should be checked against the checksum and then it will be decompressed.\n1 2 $ sha512sum FreeBSD-10.2-STABLE-arm-armv6-RPI-B-20151229-r292855.img.xz 19e051736e7112e7a0fd699e0d404cde35000b774b3c5fffbd03be35850b0efa4309d9269dd5666d326fd8c8a74ecea9d9b1cf2de28e44cc352e0e91f5e1544f FreeBSD-10.2-STABLE-arm-armv6-RPI-B-20151229-r292855.img.xz If the checksum is equal to the value shown on the Download page, then decompress it.\n1 $ unxz FreeBSD-10.2-STABLE-arm-armv6-RPI-B-20151229-r292855.img.xz The result is an IMG file, that should be copied to the SD card.\n1 2 3 4 # dd if=FreeBSD-10.2-STABLE-arm-armv6-RPI-B-20151229-r292855.img of=/dev/sdb bs=4M 120+0 records in 120+0 records out 503316480 bytes (503 MB) copied, 34.9966 s, 14.4 MB/s A few files need to be replaced on the card in order to make it boot on the Raspberry Pi Zero.\nMaking the SD Card bootable on the Zero The easiest way now is to remove and reattach the card. After reattaching, a partition is mounted (if not, it needs to be mounted manually).\n1 2 3 4 5 6 # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT … sdb 8:16 1 29.8G 0 disk ├─sdb1 8:17 1 17M 0 part /run/media/user/MSDOSBOOT └─sdb2 8:18 1 463M 0 part The MSDOSBOOT partition contains the following files:\n1 2 3 $ ls BOOTCODE.BIN FIXUP_CD.DAT RPI.DTB START.ELF UBLDR.BIN CONFIG.TXT FIXUP.DAT START_CD.ELF UBLDR U-BOOT.IMG The following files need to be deleted:\n1 $ rm BOOTCODE.BIN FIXUP_CD.DAT FIXUP.DAT START_CD.ELF START.ELF The new files need to be downloaded from the raspberrypi/firmware Github repo. The files then could be copied to the card.\n1 $ cp bootcode.bin start.elf start_cd.elf fixup.dat fixup_cd.dat /run/media/user/MSDOSBOOT/ The card can now be removed and inserted into the Raspberry Pi Zero.\n1 2 # sync # umount /run/media/user/MSDOSBOOT Booting FreeBSD The first boot takes a lot of time, because the second partition gets resized. After booting up, it is possible to log in over SSH, or locally. The default user and password combos are:\nfreebsd / freebsd\nroot / root\ndmesg on the Raspberry Pi Zero 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2015 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.2-STABLE #0 r292855: Tue Dec 29 16:17:05 UTC 2015 root@releng1.nyi.freebsd.org:/usr/obj/arm.armv6/usr/src/sys/RPI-B arm FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 VT: init without driver. CPU: ARM1176JZ-S rev 7 (ARM11J core) Supported features: ARM_ISA THUMB2 JAZELLE ARMv4 Security_Ext WB enabled LABT branch prediction enabled 16KB/32B 4-way instruction cache 16KB/32B 4-way write-back-locking-C data cache real memory = 503312384 (479 MB) avail memory = 483176448 (460 MB) random device not loaded; using insecure entropy random: \u0026lt;Software, Yarrow\u0026gt; initialized kbd0 at kbdmux0 ofwbus0: \u0026lt;Open Firmware Device Tree\u0026gt; simplebus0: \u0026lt;Flattened device tree simple bus\u0026gt; mem 0x20000000-0x20ffffff on ofwbus0 cpulist0: \u0026lt;Open Firmware CPU Group\u0026gt; on ofwbus0 cpu0: \u0026lt;Open Firmware CPU\u0026gt; on cpulist0 bcm2835_cpufreq0: \u0026lt;CPU Frequency Control\u0026gt; on cpu0 intc0: \u0026lt;BCM2835 Interrupt Controller\u0026gt; mem 0xb200-0xb3ff on simplebus0 systimer0: \u0026lt;BCM2835 System Timer\u0026gt; mem 0x3000-0x3fff irq 8,9,10,11 on simplebus0 Event timer \u0026#34;BCM2835 Event Timer 3\u0026#34; frequency 1000000 Hz quality 1000 Timecounter \u0026#34;BCM2835 Timecounter\u0026#34; frequency 1000000 Hz quality 1000 bcmwd0: \u0026lt;BCM2708/2835 Watchdog\u0026gt; mem 0x10001c-0x100027 on simplebus0 gpio0: \u0026lt;BCM2708/2835 GPIO controller\u0026gt; mem 0x200000-0x2000af irq 57,59,58,60 on simplebus0 gpio0: read-only pins: 46,47,48,49,50,51,52,53. gpio0: reserved pins: 48,49,50,51,52,53. gpioc0: \u0026lt;GPIO controller\u0026gt; on gpio0 gpiobus0: \u0026lt;OFW GPIO bus\u0026gt; on gpio0 gpioled0: \u0026lt;GPIO led\u0026gt; at pin(s) 16 on gpiobus0 iichb0: \u0026lt;BCM2708/2835 BSC controller\u0026gt; mem 0x205000-0x20501f irq 61 on simplebus0 iicbus0: \u0026lt;OFW I2C bus\u0026gt; on iichb0 iic0: \u0026lt;I2C generic I/O\u0026gt; on iicbus0 iichb1: \u0026lt;BCM2708/2835 BSC controller\u0026gt; mem 0x804000-0x80401f irq 61 on simplebus0 iicbus1: \u0026lt;OFW I2C bus\u0026gt; on iichb1 iic1: \u0026lt;I2C generic I/O\u0026gt; on iicbus1 spi0: \u0026lt;BCM2708/2835 SPI controller\u0026gt; mem 0x204000-0x20401f irq 62 on simplebus0 spibus0: \u0026lt;OFW SPI bus\u0026gt; on spi0 bcm_dma0: \u0026lt;BCM2835 DMA Controller\u0026gt; mem 0x7000-0x7fff,0xe05000-0xe05fff irq 24,25,26,27,28,29,30,31,32,33,34,35,36 on simplebus0 mbox0: \u0026lt;BCM2835 VideoCore Mailbox\u0026gt; mem 0xb880-0xb8bf irq 1 on simplebus0 sdhci_bcm0: \u0026lt;Broadcom 2708 SDHCI controller\u0026gt; mem 0x300000-0x3000ff irq 70 on simplebus0 mmc0: \u0026lt;MMC/SD bus\u0026gt; on sdhci_bcm0 uart0: \u0026lt;PrimeCell UART (PL011)\u0026gt; mem 0x201000-0x201fff irq 65 on simplebus0 uart0: console (115200,n,8,1) dwcotg0: \u0026lt;DWC OTG 2.0 integrated USB controller\u0026gt; mem 0x980000-0x99ffff irq 17 on simplebus0 usbus0 on dwcotg0 fb0: \u0026lt;BCM2835 VT framebuffer driver\u0026gt; on ofwbus0 Timecounters tick every 10.000 msec usbus0: 480Mbps High Speed USB v2.0 bcm2835_cpufreq0: ARM 700MHz, Core 250MHz, SDRAM 400MHz, Turbo OFF ugen0.1: \u0026lt;DWCOTG\u0026gt; at usbus0 uhub0: \u0026lt;DWCOTG OTG Root HUB, class 9/0, rev 2.00/1.00, addr 1\u0026gt; on usbus0 mmcsd0: 32GB \u0026lt;SDHC 00000 1.0 SN 999A04A8 MFG 03/2015 by 27 SM\u0026gt; at mmc0 41.6MHz/4bit/65535-block fb0: 640x480(0x0@0,0) 16bpp fb0: pitch 1280, base 0x5eb64000, screen_size 614400 fbd0 on fb0 VT: initialize with new VT driver \u0026#34;fb\u0026#34;. random: unblocking device. Root mount waiting for: usbus0 uhub0: 1 port with 1 removable, self powered ugen0.2: \u0026lt;vendor 0x0b95\u0026gt; at usbus0 Trying to mount root from ufs:/dev/ufs/rootfs [rw]... warning: no time-of-day clock registered, system time will not be set accurately GEOM_PART: mmcsd0s2 was automatically resized. Use `gpart commit mmcsd0s2` to save changes or `gpart undo mmcsd0s2` to revert them. axe0: \u0026lt;vendor 0x0b95 product 0x772b, rev 2.00/0.01, addr 2\u0026gt; on usbus0 miibus0: \u0026lt;MII bus\u0026gt; on axe0 ukphy0: \u0026lt;Generic IEEE 802.3u media interface\u0026gt; PHY 16 on miibus0 ukphy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow ue0: \u0026lt;USB Ethernet\u0026gt; on axe0 ue0: Ethernet address: 00:0e:c6:e0:0d:84 ue0: link state changed to DOWN ue0: link state changed to UP Thank you for reading!\n","permalink":"https://tetragir.com/posts/2016/01/booting-freebsd-on-a-raspberry-pi-zero/","summary":"Luckily, I was able to get a Raspberry Pi Zero not so long ago. Naturally, my first move was to try to boot FreeBSD on it. Although there is no official support for the Zero on FreeBSD, given that the hardware is very similar to the Raspberry Pi A model, I was able to boot it. The process is quite simple.","title":"Booting FreeBSD on a Raspberry Pi Zero"},{"content":" This post was first published on July 7, 2015 ! Its contents might be outdated!\nIntroduction I\u0026rsquo;m not sure since when, but bhyve now supports libvirt. That\u0026rsquo;s great news, given that libvirt is a universal tool which helps in managing VMs. There are quite a few hypervisors compatible with libvirt, for example KVM, Xen, Linux containers (LXC) and a bunch of others. On the other hand, there are only a few graphical fronteds compatible with it, for example Virtual Machine Manager (virt-manager), which is a general purpose VM management tool. This can be installed to a computer and the hypervisors can be managed from there. Virt-manager is generally good for a few hypervisors, with 10-20 VMs, but managing a cloud infrastructure is quite a challenge with it. There a few tools compatible with libvirt, which are targeted at clouds or greater deployment.\nIn this article, I will cover the Virtual Machine Manager (virt-manager). This tool allows for the basic management of VMs, such as installing, starting, deleting etc. The goal of this document is to introduce the installation and configuration of bhyve to a FreeBSD host, libvirt, and the virt-manager to a remote computer, then to explain how you can connect from the virt-manager to the remote host. This setup allows the management of VMs remotely. This article is customized for CentOS guest installation as a guest, however other Linux guests could be installed and used in a similar way.\nAs its current state, libvirt is not fully compatible with bhyve and therefore interacting with VMs with libvirt is not possbile. However, this article provides instructions for installing and managing VMs with bhyve itself.\nAdditional informations for this article I use the following setup:\nFreeBSD 10.1-RELEASE-p13 - as the host computer Arch Linux current - as the remote computer These two computers are on one network bhyve So what is exactly bhyve? From the offical website:\nbhyve, the \u0026ldquo;BSD hypervisor\u0026rdquo; is a hypervisor/virtual machine manager developed on FreeBSD and relies on modern CPU features such as Extended Page Tables (EPT) and VirtIO network and storage drivers.\nIt has been released on 20th January, 2014, so it\u0026rsquo;s a quite new, \u0026ldquo;shiny\u0026rdquo; technology. Therefore it is possible that a few bugs remain, or some feature do not work yet. If you encounter some, please report it. bhyve supports FreeBSD, OpenBSD, NetBSD and Linux distributions as a guest. Bhyve gets loaded and acts like KVM. This means, that bhyve itself is loaded into the kernel and that allows it to provide good performance to the virtual machines.\nIn order to boot Linux as a guest, it is important to have the sysutils/grub2-bhyve installed.\nInstalling and using bhyve bhyve is by default installed on FreeBSD, it just needs to be loaded. Load the bhyve kernel module so:\n1 kldload vmm In order to load automatically while booting up the system. It appends a new line to the /boot/loader.conf file:\n1 echo \u0026#39;vmm_load=\u0026#34;YES\u0026#34;\u0026#39; \u0026gt;\u0026gt; /boot/loader.conf Then, build sysutils/grub2-bhyve from ports:\n1 portmaster sysutils/grub2-bhyve Or install the package:\n1 pkg install sysutils/grub2-bhyve To get further information about bhyve:\n1 man bhyve Installing and starting a CentOS guest If the guest needs to be be connected to the network, a tap interface and a bridge between the physical interface and the tap has to be created. My physical interface is re0, it is important to check and correct the commands accordingly.\n1 2 3 4 5 6 ifconfig tap0 create sysctl net.link.tap.up_on_open=1 net.link.tap.up_on_open: 0 -\u0026gt; 1 ifconfig bridge0 create ifconfig bridge0 addm re0 addm tap0 ifconfig bridge0 up Create the file, which will be used as the disk for the guest:\n1 truncate -s 8G centos.img A Device map has to be created in order to clarify, which file should be mapped to which virtual device. In the example, both the centos.img, the device.map file and the CentOS-7-x86_64-DVD-1503-01.iso (which is the CentOS installer) was placed in one folder. The content of the device.map file here is:\n1 2 (hd0) ./centos.img (cd0) ./CentOS-7-x86_64-DVD-1503-01.iso To start up a Linux guest, two steps must be taken. First, point GRUB to the kernel and initrd, and then the guest can be started. The following line starts the GRUB, points to the previously created device.map file, selects the root device, allocates memory and assigns a name.\n1 grub-bhyve -m device.map -r cd0 -M 2048 centos Now the GRUB needs to be pointed to the vmlinuz and initrd files. First, check which devices are present:\n1 2 grub\u0026gt; ls (hd0) (cd0) (cd0,msdos2) (host) Check, which files are present in the CD:\n1 2 3 grub\u0026gt; ls (cd0)/ CentOS_BuildTag EFI/ EULA GPL images/ isolinux/ LiveOS/ Packages/ repodata/ RPM -GPG-KEY-CentOS-Testing-7 RPM-GPG-KEY-CentOS-7 TRANS.TBL In the isolinux folder:\n1 2 3 grub\u0026gt; ls (cd0)/isolinux boot.cat boot.msg grub.conf initrd.img isolinux.bin isolinux.cfg memtest splash .png TRANS.TBL upgrade.img vesamenu.c32 vmlinuz Then locate vmlinuz and initrd:\n1 2 grub\u0026gt; linux (cd0)/isolinux/vmlinuz grub\u0026gt; initrd (cd0)/isolinux/initrd.img Boot the installer:\n1 grub\u0026gt; boot The Linux kernel is now loaded, so it is possible to start the guest.\n1 bhyve -AI -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,./centos.img -s 4:0,ahci-cd,./CentOS-7-x86_64-DVD-1503-01.iso -l com1,stdio -c 4 -m 2048M centos CentOS CLI installer configured\nIf everything went right, the CentOS CLI installer has been started. After being installed, the instance has to be destroyed!\n1 bhyvectl --destroy --vm=centos Then the installed guest can be started:\n1 grub-bhyve -m device.map -r hd0,msdos1 -M 2048M centos Also the Linux guest is installed, and a correct grub.cfg is present.\n1 2 3 grub\u0026gt; ls (hd0,msdos1)/grub2/ themes/ device.map i386-pc/ locale/ fonts/ grubenv grub.cfg grub\u0026gt; configfile (hd0,msdos1)/grub2/grub.cfg Start the VM (without the installer ISO):\n1 bhyve -AI -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,./centos.img -l com1,stdio -c 4 -m 2048M centos The config file, installed by the CentOS installer, applies the settings to GRUB and the choose option appears. When the OS is loaded:\n1 2 uname -a Linux centos.local 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux The previosly created tap0 interface is present in the guest, and it is connected to the network. After the network is configured, the VM is reachable over SSH. After shutting down the guest, it is important to destroy the instance!\n1 bhyvectl --destroy --vm=centos In order to find out, which VMs are running, the /dev/vmm/ shoud be checked.\n1 ls /dev/vmm/ Installing and running FreeBSD as a guest Naturally, it is possible to run FreeBSD as a guest. That\u0026rsquo;s a bit easier task than running Linux. A disk image file is needed to be created:\n1 truncate -s 16G freebsd.img Then running the installer from the downloaded FreeBSD installer ISO with a built in script:\n1 sh /usr/share/examples/bhyve/vmrun.sh -c 4 -m 1024M -t tap0 -d guest.img -i -I FreeBSD-10.1-RELEASE-amd64-dvd1.iso freebsd This will start up the guest, booting from the ISO and showing the guest on the terminal. From here, the installation could be performed.\nlibvirt As I have mentioned before, libvirt is a very powerful tool that allows us to interact with hypervisors and management consoles. It is compatible with a bunch of hypervisors, and also with a lot of management tools. It not just makes the local management easier, but with a few hypervisors, it can be used remotely over SSH.\nInstalling libvirt on FreeBSD is pretty straightforward. devel/libvirt has to be either compiled from ports, or installed from packages.\nCompiling from source:\n1 portmaster devel/libvirt It is important to configure libvirt with bhyve support! Installing from packages:\n1 pkg install devel/libvirt libvirt provides a command line tool, called \u0026ldquo;virsh\u0026rdquo; to interact with hypervisors. In order to start libvirt, the daemon has to be started:\n1 service libvirtd onestart To start the libvirtd service automatically, when the host boots up, the following line needs to be added to the /etc/rc.conf file:\n1 libvirtd_enable=\u0026#34;YES\u0026#34; [!NOTE] Unfortunately starting a VM (or domain, with virsh syntax) with virsh, freezes the libvirt daemon and the VM actually does not start. In other words, at its current state, it is not possible to use bhyve with libvirt. I hope that this bug will be solved soon. Please also note, that in the example, FreeBSD 10.1-RELEASE-p13 is used, and a possible fix could have been submitted and could be present in 11-CURRENT!\nTo start the instance with virsh, a centos.xml file has been created in the folder /usr/local/etc/libvirt/qemu with contents stated on the website of libvirt. Then the file has to be initialized:\n1 virsh define centos.xml When it is initialized, the VM could be started:\n1 virsh start centos This is where it freezes. Starting a VM remotely with virt-manager will eventually work, when this issue has been fixed.\nvirt-manager The Virtual Machine Manager (virt-manager) is an easy-to-use interface for managing VMs. It is compatible with libvirt and enables the user to manage bhyve (and a bunch of other hypervisors) remotely, or locally. The developer of virt-manager is Red Hat, and the first stable version has been released on 7th of September, 2014. I have been using this application for a long time to interact with KVM, and I find it a very handy tool.\nDespite the fact that its not working as it\u0026rsquo;s current state with bhyve, here is the guide on how to continue. Because virt-manager interacts with libvirt and libvirt is unable to start a VM on bhyve, starting up a VM from virt-manager is not working. The first step is to install the virt-manager with the package manager of the distribution. On Arch Linux:\n1 pacman -S virt-manager Once installed, it is possible to connect to the bhyve running on the FreeBSD.\nThe connection has been added to the virt-manager\nSummary Despite the fact, that bhyve and libvirt have a few bugs, and do not work right now, my hopes are high conserning this combo. I find FreeBSD a very good operating system, which has just needed a fine hypervisor. I hope that the developers of bhyve and libvirt will find the problem soon and will fix it. Until that, it is possible to use bhyve from command line and access the guests from there until the SSH has been configured.\nComments It is important for me to stress, that this article is written to inform, and to be a guide to accomplish something. I am not responsible for any data losses, or for anything that goes wrong. It is your responsibility to understand the meaning of the commands and to determine the outcome.\nIt is also very important to have a backup of everything.\nThanks, sources I used I would like to thank everybody, who contributed to bhyve, libvirt and virt-manager, also the FreeBSD community, RedHat and everyone else who made this possible. I would like also to thank to artsies for proofreading this document. For this article, I used the following sources:\nFreeBSD Handbook - FreeBSD as a Host with bhyve libvirt.org - Bhyve driver FreeBSD Wiki - bhyve Wikipedia - Virtial Machine Manager Thank you for reading!\n","permalink":"https://tetragir.com/posts/2015/07/using-bhyve-on-freebsd/","summary":"I wanted to write an article about bhyve for a long time now, and fortunately I recently had the time to do just that. Bhyve has the potential to became a very sophisticated, and advanced hypervisor.","title":"Using bhyve on FreeBSD"}]