Use of Git and GitHub for software development (the basics)

 

Visual Code interface with Git plugin integrated.

Assumes Git is installed via: https://git-scm.com/downloads and that you have an account at https://github.com/

Clone an online GitHub repository to local machine

Open "Git Bash" and navigate to the directory where you would like to save the repository locally. Type:

git clone https://github.com/<USERNAME>/test_repo.git

This will clone the repository to your local machine:
Cloning into 'test_repo'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (3/3), 592 bytes | 5.00 KiB/s, done.

Edit a file and then commit changes to the GitHub repository

E.g. On your local machine edit the contents of README.md. Then using Git Bash type:

git add README.md

git commit -am "commit message" 

git push

Adding existing local project/package to Github

Create a new repository on GitHub e.g. "test_repo" and make it Private
To avoid errors, do not initialize the new repository with README, license, or gitignore files.
You can add these files after your project has been pushed to GitHub.

git init

git add .

git commit -m "First commit" 

At the top of your GitHub repository's Quick Setup page, click to copy the remote repository (https) URL.

git remote add origin https://github.com/<USERNAME>/test_repo.git

git remote -v

git push origin master

Git configuration for use with proxy - Optional

Open "Git Bash" shell program (installed as part of Git) and type:

git config --global http.proxy http://USERNAME:PASSWORD@PROXYADDRESS:PROXYPORT
git config --global https.proxy http://USERNAME:PASSWORD@PROXYADDRESS:PROXYPORT

freeNAS as a RAID-Z / iSCSI file storage server

freeNAS web interface for storage management.

FreeNAS is a free and open-source network-attached storage operating system based on FreeBSD and the OpenZFS file system. 

Using the e.g. the Dell PowerEdge RAID Controller H310 (but flashed to IT mode).

Install freeNAS

Download the ISO image file from the FreeNAS website and burn it to a CD. Put the CD into the server as well as the flash disk and switch it on. Make sure that the BIOS is set to boot from CD. If it isn’t, you need to enter into the BIOS and configure it to boot from CD. Follow install steps and select the flash drive that you want to install freeNAS on. You will be prompted to set a password for the "root" user.
Once installation is complete take the CD out and reboot the system.

Initial boot

When you boot into FreeNAS the Console Setup will appear at the end of the boot process. This Console Setup menu can be used to administer the system should the web interface be inaccessible.
NOTE: you can access the Console Setup menu from freeBSD by typing /etc/netcli from shell. You can disable the Console Setup menu by unchecking the “Enable Console Menu” in System ‣ Advanced settings in the freeNAS web interface.

Set a static IP address

From the Console Setup menu:

• Select "Configure Network Interfaces" (1)
• Select "bce0" (1)
• Select n (for "Remove the current settings of this interface?...)
• Select n (for "Configure interface for DHCP")
• Select y (for "Configure IPv4")
• Enter the Interface name:bce0
• Enter the IPv4 Address: 10.140.79.211
• Enter the Netmask: 255.255.255.0
• Select n (for "Configure UPv6")

Access freeNAS web interface

Use web browers: https://10.140.79.211 or https://hostname e.g. where hostname = freenas

Enable SSH connection to server

NOTE: It is important to use the GUI (or the Console Setup menu) for all configuration changes. FreeNAS uses a configuration database to store its settings. While it is possible to use the command line to modify your configuration, changes made at the command line are not written to the configuration database. This means that any changes made at the command line will not persist after a reboot and will be overwritten by the values in the configuration database.
The terminal can be access via the web interface but to use SSH:

Open /etc/rc.conf and insert this at the end sshd_enable="YES" 
    run /etc/rc.d/sshd start
    Type a full line of junk as requested to generate the keys
    OK sshd is now running but you won't be able to login into it with just a user name and password
    nano /etc/ssh/sshd_config
    change the line "PasswordAuthentication no" to PasswordAuthentication yes and remove the # at the start of line
    If you want to login as root PermitRootLogin yes
    Restart sshd by running /etc/rc.d/sshd restart
    Check status with /etc/rc.d/sshd status

This can be done by logging into freeNAS in web browser (username: root) and selecting "Shell" in the left hand menu.

Configure for use

Access the web interface.

1. Set up a Storage Pool and RAID

• Select Storage > Pools > ADD > Create new pool > CREATE POOL
• Enter a Pool name (e.g. "MedPhysStore" and add the available disks by selecting with check box and using the right arrow to move them to Data VDevs.
• Under this select Raid-z (Note: RAIDZ1: requires at least three disks, RAIDZ2: requires at least four disks, RAIDZ3: requires at least five disks). RAID-Z is similar to RAID 5 but "faster": https://en.wikipedia.org/wiki/Non-standard_RAID_levels#RAID-Z
• Select "CREATE" (Note: a Warning will pop up about the contents of all disks being erased, check "Confirm" and click "CREATE POOL") and the pool is created. It should indicate HEALTHY with 0% Used.

2. Set up an iSCSI share

Create a Zvol

• After you create your pool, click on the 3 dots next to it and select “Add Zvol”.
• It’s recommended to not exceed 50% of your storage size for your iSCSI share so for a pool of size X (e.g. HEALTHY: Y KiB (0%) Used / X TiB Free) make the iSCSI share X/2 TiB.
• Enter the Zvol name (e.g. MedPhysStore) and Size (i.e. X/2 GiB) for this zvol and Save

Create a Portal

• Enable the iSCSI service: Click on Services and enable iSCSI and check "Start Automatically".
• Go to Sharing > Block Shares (iSCSI) > Portals and click ADD.
• Select the IP addresses to be listened on by the portal. The address 0.0.0.0 can be selected to listen on all IPv4 addresses (the default iSCSI TCP port number is 3260)
• SAVE

Add a Target

• Click on Targets and ADD.
• Enter a Target Name (e.g. medphystarget) and Portal Group ID (e.g. 1) and SAVE.

Add an Extent

• An Extent is a storage unit shared through iSCSI. You can use a physical device or a file as Extent.
• Select "Extents" and "Add Extent".
• Enter an "Extent name" and select the "Device" as the Storage pool/Zvol created above and check "Enabled" and SAVE.

Add Associated Targets

• Click on Associated Targets and ADD.
• Select the Target and Extent created above and SAVE.

Access the iSCSI share from a Window PC

• Open Control Panel\All Control Panel Items\Administrative Tools\iSCSI Initiator
• Enter the Target IP of the freeNAS controller (e.g. 10.140.79.211) and click "Quick Connect...".
• The available iSCSI share should appear in the "Discovered targets" box.
• Go to the Volume and Devices tab and click Auto Configure.
• Open "Computer Management" and expand Storage and click on "Disk Management"
• A "Initialize Disk" pop-up will appear. Select GPT and "OK".
• Right-click the disk volume for the share, select a drive letter (e.g. F) and then choose a volume formatting option.
• The disk should now be available for use from within Windows File Explorer.

Environmental monitoring with Nagios, Raspberry Pi 3 and BME680 sensor

Raspberry Pi 3 with BME680 attached (left) and Nagios web interface (above and right)

Nagios, now known as Nagios Core, is a free and open-source computer-software application that monitors systems, networks and infrastructure. Nagios offers monitoring and alerting services for servers, switches, applications and services

Raspberry Pi's are a series of small cheap single-board computers.

Below is Nagios Core  / NRPE plugin (written in Python 3) for monitoring of environment (e.g. in a Server room or elsewhere) using a Raspberry Pi (3) and the BME 680 sensor attached:

Code on GitHub:

https://github.com/tuathano/check_temp_pres_hum_gas/blob/master/check_temp_pres_hum_gas.py

Nagios Core:

https://www.nagios.org/downloads/nagios-core/ 

The BME 680 sensor which can be attached to the Raspberry Pi 3:

https://www.adafruit.com/product/3660

# check_temp_pres_hum_gas.py

#

# python 3 script to poll BME680 enviromental sensor and return Nagios exit codes

# T. O'Shea / R. Trouncer 2020

#

# v1: 22/01/2020: Initial version

# v1.1: 24/01/2020: Minor edit; changed - to : in output string to avoid confusion with minus sign.

# v2: 28/01/2020: Added median filtering of raw sensor data. User can input 2 additional arguments from command

# line to modify the total length and time interval (s) between points for the filter.

# v3: 29/01/2020: Combine all sensor checks into one serial god-awful check as the BME680 chip cannot cope with lots of seperate polling.

# v3.1: 30/01/2020: Minor bug fix in WARNING condition

#

# Nagios exit codes:

# 0 - Service is OK

# 1 - Service is WARNING

# 2 - Service is in a CRITICAL status

# 3 - Service status is UNKNOWN

#

# Usage example:

#

# check_temp_pres_hum_gas.py <selected_meas> (<crit_min> <crit_max> <warn_min> <warn_max> x4) <filt_pts> <filt_int>

#

# e.g. check_temp_pres_hum_gas.py 12 28 18 25 10 100 20 80 10000 10000000 500000 9000000 900 1070 920 1060 5 1

#

# The first set of crit_min/max and warn_min/max values are for temperature, then humidity, then gas and finally pressure.

# <filt_pts> is the number of points to use when median filtering the raw data and <filt_int> is the time interval (in seconds) between the points.

# Required libraries

import sys

import board

import busio

import adafruit_bme680

import time

import statistics

#!/usr/bin/env python