Posts tagged: cli

Authentication from cli tools can be a bit of a bear, and I have to look it up every time. This is my reference guide for future me to remember how to easily do it.

I set up a fastapi server running on port 8000, it uses a basic auth with waylonwalker as the username and asdf as the password. The server follows along with what comes out of the docs. I have it setup to take basic auth, form username and password, or a bearer token for authentication.

curl #

The og of command line url tools.

# basic auth
curl -u 'waylonwalker:asdf' -X POST localhost:8000/token
# basic auth with password prompt
curl -u 'waylonwalker' -X POST localhost:8000/token
# token
curl -H 'Authorization: bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ3YXlsb253YWxrZXIiLCJleHAiOjE3MDI5NTI2MDJ9.GeYNt7DNal6LTiPoavJnqypaMt4vYeriXdq5lqu1ILg' -X POST localhost:8000/token

wget #

My go to if I want the result to go into a file.

# basic auth
wget -q -O - --auth-no-challenge --http-user=waylonwalker --http-password=asdf --post-data '' localhost:8000/token

# token
wget -q -O - --header="Authorization: bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ3YXlsb253YWxrZXIiLCJleHAiOjE3MDI5NTI2MDJ9.GeYNt7DNal6LTiPoavJnqypaMt4vYeriXdq5lqu1ILg" -O - --post-data '' localhost:8000/token

httpx #

An http client written in python, primarilty used with the python api, but has a nice cli.

# install
python3 -m pip install httpx

# basic auth
httpx -m POST --auth waylonwalker asdf http://localhost:8000/token

# basic auth with password prompt
httpx -m POST --auth waylonwalker - http://localhost:8000/token

# token
httpx -m POST --headers="Authorization" "bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ3YXlsb253YWxrZXIiLCJleHAiOjE3MDI5NTI2MDJ9.GeYNt7DNal6LTiPoavJnqypaMt4vYeriXdq5lqu1ILg" http://localhost:8000/token

httpie #

A modern http client written in python.

# install
python3 -m pip install httpie

# basic auth
http POST localhost:8000/token -a waylonwalker:asdf

# basic auth with password prompt
http POST localhost:8000/token -a waylonwalker

# token
http POST localhost:8000/token -A bearer -a eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ3YXlsb253YWxrZXIiLCJleHAiOjE3MDI5NTI2MDJ9.GeYNt7DNal6LTiPoavJnqypaMt4vYeriXdq5lqu1ILg

httpie with plugin #

# install
python3 -m pip install httpie-credential-store
# usage
http POST localhost:8000/token -A creds

httpie prompt #

http-prompt comes from the httpie org, and has an interactive cli interface into apis. You can even specify a spec file to autocomplete on api methods.

http-prompt localhost:8000 --auth waylonwalker:asdf --spec openapi.json

jpillora/installer is the install script generator I have been looking for. It downloads binaries for your machine from GitHub releases and unzips them for you. It grabs the latest release, so you can easily update them. I have tried scripting these installs in the past and struggled to consistently get the latest version for every package and unpack it correctly.

Also these pre-compiled binaries install rediculously fast compared to building them from source.

Check out some example links.

opening in a browser will show metadata

https://i.jpillora.com/serve

If you pass in script=true it will instead return the install script as it would by default through curl.

https://i.jpillora.com/serve?script=true

Use it to install neovim #

All you need to do to generate an install script is to pass in the GitHub repo slug with the org.

curl https://i.jpillora.com/neovim/neovim | bash

The shell script that it generates for neovim looks like this.

#!/bin/bash
if [ "$DEBUG" == "1" ]; then
    set -x
fi
TMP_DIR=$(mktemp -d -t jpillora-installer-XXXXXXXXXX)
function cleanup {
    rm -rf $TMP_DIR > /dev/null
}
function fail {
    cleanup
    msg=$1
    echo "============"
    echo "Error: $msg" 1>&2
    exit 1
}
function install {
    #settings
    USER="neovim"
    PROG="neovim"
    ASPROG=""
    MOVE="false"
    RELEASE="stable"
    INSECURE="false"
    OUT_DIR="$(pwd)"
    GH="https://github.com"
    #bash check
    [ ! "$BASH_VERSION" ] && fail "Please use bash instead"
    [ ! -d $OUT_DIR ] && fail "output directory missing: $OUT_DIR"
    #dependency check, assume we are a standard POISX machine
    which find > /dev/null || fail "find not installed"
    which xargs > /dev/null || fail "xargs not installed"
    which sort > /dev/null || fail "sort not installed"
    which tail > /dev/null || fail "tail not installed"
    which cut > /dev/null || fail "cut not installed"
    which du > /dev/null || fail "du not installed"
    #choose an HTTP client
    GET=""
    if which curl > /dev/null; then
        GET="curl"
        if [[ $INSECURE = "true" ]]; then GET="$GET --insecure"; fi
        GET="$GET --fail -# -L"
    elif which wget > /dev/null; then
        GET="wget"
        if [[ $INSECURE = "true" ]]; then GET="$GET --no-check-certificate"; fi
        GET="$GET -qO-"
    else
        fail "neither wget/curl are installed"
    fi
    #debug HTTP
    if [ "$DEBUG" == "1" ]; then
        GET="$GET -v"
    fi
    #optional auth to install from private repos
    #NOTE: this also needs to be set on your instance of installer
    AUTH="${GITHUB_TOKEN}"
    if [ ! -z "$AUTH" ]; then
        GET="$GET -H 'Authorization: $AUTH'"
    fi
    #find OS #TODO BSDs and other posixs
    case `uname -s` in
    Darwin) OS="darwin";;
    Linux) OS="linux";;
    *) fail "unknown os: $(uname -s)";;
    esac
    #find ARCH
    if uname -m | grep -E '(arm|arch)64' > /dev/null; then
        ARCH="arm64"

        # no m1 assets. if on mac arm64, rosetta allows fallback to amd64
        if [[ $OS = "darwin" ]]; then
            ARCH="amd64"
        fi

    elif uname -m | grep 64 > /dev/null; then
        ARCH="amd64"
    elif uname -m | grep arm > /dev/null; then
        ARCH="arm" #TODO armv6/v7
    elif uname -m | grep 386 > /dev/null; then
        ARCH="386"
    else
        fail "unknown arch: $(uname -m)"
    fi
    #choose from asset list
    URL=""
    FTYPE=""
    case "${OS}_${ARCH}" in
    "linux_amd64")
        URL="https://github.com/neovim/neovim/releases/download/stable/nvim-linux64.tar.gz"
        FTYPE=".tar.gz"
        ;;
    "darwin_amd64")
        URL="https://github.com/neovim/neovim/releases/download/stable/nvim-macos.tar.gz"
        FTYPE=".tar.gz"
        ;;
    *) fail "No asset for platform ${OS}-${ARCH}";;
    esac
    #got URL! download it...
    echo -n "Downloading"
    echo -n " $USER/$PROG"
    if [ ! -z "$RELEASE" ]; then
        echo -n " $RELEASE"
    fi
    if [ ! -z "$ASPROG" ]; then
        echo -n " as $ASPROG"
    fi
    echo -n " (${OS}/${ARCH})"

    echo "....."

    #enter tempdir
    mkdir -p $TMP_DIR
    cd $TMP_DIR
    if [[ $FTYPE = ".gz" ]]; then
        which gzip > /dev/null || fail "gzip is not installed"
        bash -c "$GET $URL" | gzip -d - > $PROG || fail "download failed"
    elif [[ $FTYPE = ".tar.bz" ]] || [[ $FTYPE = ".tar.bz2" ]]; then
        which tar > /dev/null || fail "tar is not installed"
        which bzip2 > /dev/null || fail "bzip2 is not installed"
        bash -c "$GET $URL" | tar jxf - || fail "download failed"
    elif [[ $FTYPE = ".tar.gz" ]] || [[ $FTYPE = ".tgz" ]]; then
        which tar > /dev/null || fail "tar is not installed"
        which gzip > /dev/null || fail "gzip is not installed"
        bash -c "$GET $URL" | tar zxf - || fail "download failed"
    elif [[ $FTYPE = ".zip" ]]; then
        which unzip > /dev/null || fail "unzip is not installed"
        bash -c "$GET $URL" > tmp.zip || fail "download failed"
        unzip -o -qq tmp.zip || fail "unzip failed"
        rm tmp.zip || fail "cleanup failed"
    elif [[ $FTYPE = ".bin" ]]; then
        bash -c "$GET $URL" > "neovim_${OS}_${ARCH}" || fail "download failed"
    else
        fail "unknown file type: $FTYPE"
    fi
    #search subtree largest file (bin)
    TMP_BIN=$(find . -type f | xargs du | sort -n | tail -n 1 | cut -f 2)
    if [ ! -f "$TMP_BIN" ]; then
        fail "could not find find binary (largest file)"
    fi
    #ensure its larger than 1MB
    #TODO linux=elf/darwin=macho file detection?
    if [[ $(du -m $TMP_BIN | cut -f1) -lt 1 ]]; then
        fail "no binary found ($TMP_BIN is not larger than 1MB)"
    fi
    #move into PATH or cwd
    chmod +x $TMP_BIN || fail "chmod +x failed"
    DEST="$OUT_DIR/$PROG"
    if [ ! -z "$ASPROG" ]; then
        DEST="$OUT_DIR/$ASPROG"
    fi
    #move without sudo
    OUT=$(mv $TMP_BIN $DEST 2>&1)
    STATUS=$?
    # failed and string contains "Permission denied"
    if [ $STATUS -ne 0 ]; then
        if [[ $OUT =~ "Permission denied" ]]; then
            echo "mv with sudo..."
            sudo mv $TMP_BIN $DEST || fail "sudo mv failed"
        else
            fail "mv failed ($OUT)"
        fi
    fi
    echo "Downloaded to $DEST"
    #done
    cleanup
}
install

Self Host Your Own #

I’d reccomend self hosting your own. This way you know that it’s consistent and unlikely to change in a way that breaks your use.

curl -s https://i.jpillora.com/installer | bash

Repos I am using installer for #

Here are the repos I am using installer for.

atuinsh/atuin
benbjohnson/litestream
bootandy/dust
BurntSushi/ripgrep
chmln/sd
cjbassi/ytop
dalance/procs
dbrgn/tealdeer
ducaale/xh
go-task/task
imsnif/bandwhich
imsnif/diskonaut
kovidgoyal/kitty
mgdm/htmlq
neovim/neovim
ogham/dog
ogham/exa
pemistahl/grex
sharkdp/bat
sharkdp/fd
sharkdp/pastel
sirwart/ripsecrets
starship/starship
topgrade-rs/topgrade
zellij-org/zellij