This is a living list of code snippets and commands that have been useful to me.

Python

Convert between CSV and JSON

https://books.agiliq.com/projects/tweetable-python/en/latest/file-conversion.html#csv-to-json

Concurrency & Threading

Dead simple way to add threading or multiprocessing. Let’s say we have a function download that downloads a file to a directory.

from os.path import join
from urllib.request import urlretrieve
def download(directory, url):
  urlretrieve(url, join(directory, url.split('/')[-1])

We can run this syncronously, with multithreading, or with multiprocessing.

1. Running syncronously

   urls = ['https://i.imgur.com/NlrZk7C.jpg', 'https://i.imgur.com/HLRBXyd.jpg']
   directory = "/tmp/"
   for url in urls:
    download(directory, url)
   

2. With Threading

   from concurrent.futures import ThreadPoolExecutor
   from functools import partial
   NUM_WORKERS = 3
   download = partial(download, directory) 
   with ThreadPoolExecutor(max_workers=NUM_WORKERS) as executor:
     results = executor.map(download, urls)
   

3. With Multiprocessing

   from concurrent.futures import ProcessPoolExecutor
   from functools import partial
   NUM_WORKERS = 3
   download = partial(download, directory) 
   with ProcessPoolExecutor(max_workers=NUM_WORKERS) as executor:
     results = executor.map(download, urls)
   

Shell

Generate list of all file extensions in a directory

find . -type f \
  | awk -F. '{print $NF}' \
  | sort \
  | uniq \
  | awk 'BEGIN { ORS=" "; print "[" }; {printf "'.%s',", $1}; END { print "]" }' && echo

Get width/height of all images in directory

find . -type f -exec sips -g pixelHeight -g pixelWidth {} \;

Spoof your MAC address

# see your current mac address
ifconfig <network interface, such as en1> | grep ether
# see other mac addresses on the network
arp -a
# set your mac address
sudo ifconfig <network interface> ether <new mac address>