Thomas Willingham

Thomas Willingham

© 2020

Code Snippets

This is a living list of code snippets and commands that have been useful to me.

Python

Convert between CSV and JSON

https://books.agiliq.com/projects/tweetable-python/en/latest/file-conversion.html#csv-to-json

Concurrency & Threading

Dead simple way to add threading or multiprocessing. Let’s say we have a function download that downloads a file to a directory.

from os.path import join
from urllib.request import urlretrieve
def download(directory, url):
  urlretrieve(url, join(directory, url.split('/')[-1])

We can run this syncronously, with multithreading, or with multiprocessing.

  1. Running syncronously
    urls = ['https://i.imgur.com/NlrZk7C.jpg', 'https://i.imgur.com/HLRBXyd.jpg']
    directory = "/tmp/"
    for url in urls:
     download(directory, url)
    
  2. With Threading
    from concurrent.futures import ThreadPoolExecutor
    from functools import partial
    NUM_WORKERS = 3
    download = partial(download, directory) 
    with ThreadPoolExecutor(max_workers=NUM_WORKERS) as executor:
      results = executor.map(download, urls)
    
  3. With Multiprocessing
    from concurrent.futures import ProcessPoolExecutor
    from functools import partial
    NUM_WORKERS = 3
    download = partial(download, directory) 
    with ProcessPoolExecutor(max_workers=NUM_WORKERS) as executor:
      results = executor.map(download, urls)
    

Shell

Generate list of all file extensions in a directory

find . -type f \
  | awk -F. '{print $NF}' \
  | sort \
  | uniq \
  | awk 'BEGIN { ORS=" "; print "[" }; {printf "'.%s',", $1}; END { print "]" }' && echo

Get width/height of all images in directory

find . -type f -exec sips -g pixelHeight -g pixelWidth {} \;

Spoof your MAC address

# see your current mac address
ifconfig <network interface, such as en1> | grep ether
# see other mac addresses on the network
arp -a
# set your mac address
sudo ifconfig <network interface> ether <new mac address>