Welcome to Software Carpentry

We will use this Etherpad to share links and snippets of code, take notes, ask and answer questions, and whatever else comes to mind.
The page displays a screen with three major parts:

* The left side holds today's notes: please edit these as we go along.
* The top right side shows the names of users who are logged in: please add your name and pick the color that best reflects your mood and personality.
* The bottom right is a real time chat window for asking questions of the instructor and your fellow learners.

To start, please add yourself to the attendee list below:



Post-Workshop Survey
https://www.surveymonkey.com/r/swc_post_workshop_v1?workshop_id=2019-08-26-genentech

Day 1 - The BASH Shell (This lesson and additional materials can be found at <http://swcarpentry.github.io/shell-novice/>)

Please open the website and the etherpad
Please also open a tab with socrative.com
Please install the data here: https://swcarpentry.github.io/shell-novice/data/data-shell.zip

pwd = present working directory
ls = list of pwd
ls -F = list files
ls -F / = list file, root directory
ls -a = list all, including hidden files and directories (e.g. files and directories that start with ".", too)
ls .. = directory above where currently sitting
cd = change directory
cd -p = return to home directory
cd .. = chg. directory one level up
cd ../.. = chg. directory two levels up
cd ../../.. = chg. directory three levels up
cd ~ = return to home directory
man ls = lists the BSD General Commands (type "q" to exit, or from ANY program)
cd / = returns to root directory
mkdir = make directory
nano filename.txt = text editor, open file "filename.txt" w/ nano text editor software
cat filename.txt = print file contents to screen
rm filename.txt = delete file
rmdir directory/ = delete directory
cp = cap completion allows for backup file to be made; cp p* namesp2 = copies all files that start w/ p in current directory to the namesp2 directory
mv "original file name" "new file name"= renames a file
rm -r "directory"/ = removes a directory (rm -r / erases whole computer!! <- forever. no backups. no undo. just sorrow and regret.)
mv "file name" directory = moves a file to a new directory
A "*" wildcard allows for any length of c haracters to be searched for as a suffix or prefix; a "?" wildcard replaces only a single character
wc = word count (lists # of file lines, # of words, and # of characters)
cntrl L clears terminal screen
wc -l *.pdb >file_lengths.txt = copies all .pdb file word count outputs and places them in a new file
sort file name = sorts text lines in a file; sort-n sorts numerically
q exits a given software program
head = lists first portion of file; tail lists last lines (add -n "#" for first/ last lines)  
history = prints command history 
> = save output of a command to a file
| = "pipe" send output of one command into the next command, e.g history | tail = print last 10 lines of command history; pipe takes std. out from one command and makes it std. in for the next command


You can save commands as textfiles (convetion is to use .sh suffix) and then run the textfile as argument to bash, e.g. "bash command.sh" or "bash command.sh arg1"
$1 = variable for first arguement you pass to a script
$@ = variable for every arguement that you pass to a script
Ctrl + C to escape a hanging command 
Use arrow keys to toggle between recently used commands

STACKOVERFLOW.com is a great online source for Q & A regarding programming

Day 1 - Python

Additional software carpentry python materials can be found at: <http://swcarpentry.github.io/python-novice-gapminder/>

Two excellent Python books, both freely available as Jupyter notebooks:
    - https://jakevdp.github.io/PythonDataScienceHandbook (data science, pandas, matplotlib)
    - https://nbviewer.jupyter.org/github/jakevdp/WhirlwindTourOfPython/blob/master/Index.ipynb (introduction)

Example data: https://swcarpentry.github.io/python-novice-gapminder/files/python-novice-gapminder-data.zip
Notes from the Python lesson: https://github.com/JoaoRodrigues/sc-teaching/blob/master/python-gapminder-aug-2019/IntroToPython.ipynb 

Opening Jupyter Lab:
    Windows: Use the "Anacondaprompt" shell instead of the usual terminal
    Mac: open a new terminal window
    Then, type in "jupyter lab" to open Jupyter s/w
    If either of these don't work, run `jupyter notebook` instead at the command line

pandas package- for loading excel files

changing cell type
code = for code (obviously); to toggle cell type to code type "y"
markdown = for notes about the code; to toggle cell type to notes type "m"
using "# " before a line will format it as a header, type shift+enter after to start a new cell
If you reassign a variable later in the code its value will be determined in the order the jupyter cells are run, not necessarily the order of the cells from top to bottom of the notebook file
file name [0:8:2] returns data for that file from length 0 to end 8th character, w/ a spacing of 2 or every other character
press shift/ return at the end of each code line to enter the line
type (variable) returns the variable type for that variable
can change variable type by specifying variable type first then add variable name eg int(age_str)
mouse over code line # in Jupyter, and press  a to add a lin e above the moused over line number; use b to add one below
index_col lists that column as the first column
data.head and .tail displays data at beginning and end of data
data.T  transposes data output
use parentheses at the end of any function command

Python Day 2

Before we start for the day:
    (1) Sign in using sheet at the front
    (1.b) Grab pink and blue stickies
    (2) Save any jupyter notebooks you have open
    (3) Open a brand new notebook


!ls within Jupyter lists files in Jupyter location; also !pwd
For loop allows for a repeated command in Jupyter (4 spac es are crucial in including items within the for loop; non-indented items are not in the for loop and will not be repeated
len(my_list) results in length of items in file; len(my_list[0]) results in length of item 0 in list
Brackets are used to enclose values or a new numerical variable
If/then command: only use "if", the "then" is implied; always end both if/then and for/next commands with a colon
Add-ons to if/then are: Else where this is an alternative if the "if" commnd is not satisfied; and Elif where this acts as an "or"
Use "def" to define a new function
Within def command, can use return to assign a value to a variable 


How to write scripts outside Jupyter Notebook
1) make sure your script imports proper modules (matplotlib, pandas, etc..)
2) Copy from the code you prototyped in Jupyter notebook to a new text file (e.g. using notepad)
2) save your script as '[name].py' (i.e. script.py) 
3) save that script to the directory with your data
4) open terminal and go to that directory
5) type "python script.py" to run it

import matplotlib.pyplot as plt 
import pandas as pd   
data = pd.read_csv('gapminder_gdp_americas.csv', index_col='country')  
def gather_years_and_gdp_cols(column_names):    
    years = []    gdp_cols = []    for cname in column_names:        
        if cname.startswith('gdpPercap'):            
            years.append(int(cname[-4:]))            
            gdp_cols.append(cname)    
        return years, gdp_cols  
        
years, gdp_cols = gather_years_and_gdp_cols(data.columns)  
# plt.legend() 
plt.title("GDP Percap Over 50 Years")  
plt.savefig("outfigure.pdf")


Git - Day 2
Additional lesson materials here: <https://swcarpentry.github.io/git-novice/>

Before we start:
      Windows Users:
            Open a new terminal session on Git Bash
       Mac Users
           Open a  brand new terminal window.


git config --global user.name  "NAME"
git config --global user.name "<YOUR NAME HERE>"
git config --global user.email "<YOUR EMAIL HERE>"
git config --global color.ui "auto"
git config --global core.editor "nano -w"
git config --global core.autocrlf input
git init establishes a directory to be associated w/ tracked storage
git file association first gets staged, then a file is committed for tracking
commit msg titles should be brief but descriptive
use touch to hide a file from being tracked and create a .gitignore file w/ *draft