I’m impressed by kedro-viz [1] from kedro-org [2].
Visualise your Kedro data and machine-learning pipelines and track your experiments.
References:
[1]: https://github.com/kedro-org/kedro-viz
[2]: https://github.com/kedro-org
Publishing rhythm
Should I switch to Zeit Now
Netlify # [1]
I have happily had my personal site waylonwalker.com [2] hosted on netlify for nearly 2 years now. In fact I have hosted about a dozen different toy projects to play with on there, 4 of which have gone far enough to get a custom domain name. They are fast to deploy and consistently do so on every git push to main.
Zeit # [3]
I have recently started playing with zeit again. I really like their cli tool, its dead simple and makes sense. I tried the netlify one early on and dont think I really gave it much of a chance. I was able to backup a site we were modifying by saving everything locally (literally control+s) and running now in the command line.
While I was on the site I realized that when switching to gatsby v2 I had deployed it to now.sh while testing, before cutting over to the updated one on netlify. It has been building every version since without issue!
It has been building every version since without issue!
What I am using # [4]
- gatsby
- forestry.io
...
git push without setting upstream
Finally after years of hand typing out a full git push --upstream my_really_long_and_descriptive_branch_name I found there is a setting to
automatcally push to the current branch. More realisitically I just did a git push let git [1] yell at me, and copying the suggestion.
git config # [2]
git config --global push.default current
This one setting will now git push to the current branch without yelling at
you that your upstream does not match your current branch. This helps me ship
chnages faster as I am constantly chnaging projects and branches.
References:
[1]: /glossary/git/
[2]: #git-config
Out of Space
This morning I logged into my machine and was nearly out of space
- 64GB miniconda3!
- 5GB conda cache
- 4GM pip cache
- 34GB docker
Find it # [1]
[2]
These are the commands that I often use to reclaim space. Its so easy to fill
up small vm’s in the cloud, or in my case today let your dev machine go way too
long without a good cleanup.
Show Remaining Space on Drives # [3]
This shows us where to start and gives a baseline of how much space we have
reclaimed.
df -h
show largest files in current directory # [4]
Next keep drilling into directories that are big and running this command to
see whats big inside of it. When you find somethign that you are willing to
part with rm -rf <directory> it and check df -h to see if you have enough
reclaimed yet.
du . -h --max-depth=1
Honestly I rarely bother unless the directory is in the GB’s of space. A super
simple filter for that is to just grep for G.
du . -h --max-depth=1 | grep G
conda # [5]
How Many? # [6]
As a first baseline...
simple click
cli tools are super handy and easy to add to your python libraries to supercharge them. Even if your library is not a cli tool there are a number of things that a cli can do to your library.
Example Ideas # [1]
Things a cli can do to enhance your library.
🆚 print version
🕶 print readme
📝 print changelog
📃 print config
✏ change config
👩🎓 run a tutorial
🏗 scaffold a project with cookiecutter
🖱 Click [2] # [3]
Click [2] is the most popular python cli tool framework for python. There are others, some old, some new comers that make take the crown. For now Click [2] is the gold standard if you want to make a powerful cli quickly. If you are dependency conscious and dont need a lot of tooling, use argparse [4].
Project Structure # [5]
.
├── setup.py
└── simple_click
├── cli.py
└── __init__.py
❯ cli.py # [6]
# simple_click/cli.py
import click
__version__ = "1.0.0"
@click.group()
def cli():
pass
@cli.command()
def version():
"""prints project version"""
click.echo(__...
Personal URL shortener with Netlify Redirects
I love using URL shorteners to easily share links without hitting character
limits, but they loose their meaning. Services like bit.ly will save my links
for me so that I can find them, but I would rather them to be easy to remember.
https://bit.ly/2ruLwQz does
not roll of the tongue so well.
301 🤸♀️ # [1]
I recently discovered a really cool feature of netlify that I have always looked past, _redirects. It is so simple cool and powerful, every netlify site should do this!
But how 🤷♀️ # [2]
simply add a _redirects file to the root of your your published site with the following format. The trick I found with my gatsby site was that it needed to be in my static directory /static/_redirects, not root. Next you just put space separated links on separate lines. #’s can be used for comments.
# netlify redirects
# from_url to_url
# Short-Blog
/blog/scli /blog/simple-click/
/blog/cmdt /blog/cmd-exe-tips/
.
.
.
# splats
/b* /blog/:splat
/n* /notes/:splat
# External
/twitter https...
szsdk [1] has done a fantastic job with quick [2]. Highly recommend taking a look.
A real QUICK Qt5 based gUI generator for ClicK
References:
[1]: https://github.com/szsdk
[2]: https://github.com/szsdk/quick
cmd.exe tips
I spend a lot of my time at the terminal for my daily work, mostly in Linux or wsl. One big reason for using wsl over cmd.exe is the ease of walking through history that fzf provides. This week we had a windows bug in a cli and I was stuck in vanilla cmd.exe 😭
> Cmder # [1]
[2]
First off if you are stuck using cmd.exe, do yourself a favor and get cmder. It makes life just a bit easier. It is super confugurable and comes with several power ups that make it a bit more enjoyable than cmd.exe.
History # [3]
F7 - Scroll through history
F8 - Search history based
Example # [4]
image [5]
.bat # [6]
The next simple technique is to save your commands into a .bat file. Any valid command ran with cmd.exe can be saved into a bat file and called again later by running it in the terminal.
save your command
use f7/f8 to get your command back add > filename.bat at the end, hit the home key and add echo to the front. Do not wrap with quotes. This is not bash.
echo python cmd_example.py ...
If you’re into interesting projects, don’t miss out on forgit [1], created by wfxr [2].
💤 A utility tool powered by fzf for using git [3] interactively.
References:
[1]: https://github.com/wfxr/forgit
[2]: https://github.com/wfxr
[3]: /glossary/git/
I’m really excited about python-c2f [1], an amazing project by grantjenks [2]. It’s worth exploring!
Cython for All with GitHub Actions
References:
[1]: https://github.com/grantjenks/python-c2f
[2]: https://github.com/grantjenks
Check out wesbos [1] and their project beginner-javascript [2].
Slam Dunk JavaScript
References:
[1]: https://github.com/wesbos
[2]: https://github.com/wesbos/beginner-javascript
If you’re into interesting projects, don’t miss out on awesome-react-components [1], created by brillout [2].
Curated List of React Components & Libraries.
References:
[1]: https://github.com/brillout/awesome-react-components
[2]: https://github.com/brillout
What is something you should have learned or understood earlier?
Mine is the python debugger. I was a long holdout thinking that print statements were sufficient. That was untill I started having errors crop up in functions that took minutes to run. The thing that I most notably wish I would have known about is post_mortem.
Example # [1]
[ins] In [4]: def repeater(msg, repeats=1):
...: "repeats messages {repeats} number of times"
...: print(f'{msg}\n' * repeats)
[ins] In [5]: repeater('hi', 3)
hi
hi
hi
[ins] In [6]: repeater('hi', 'a')
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-0ec595774c81> in <module>
----> 1 repeater('hi', 'a')
<ipython-input-4-530890de75cd> in repeater(msg, repeats)
1 def repeater(msg, repeats=1):
2 "repeats messages {repeats} number of times"
----> 3 print(f'{msg}\n' * repeats)
4
Debug with iPython/Jupyter
%debug
Vanilla Debug # [2]
import pdb
import sys
pdb.post_mortem(sys.last_traceback)
More # [3]
For more inform...
Supercharge Zsh Startup
I have been using oh-my-zsh successfully for about 2 years now. But lately my
startup time has been really bothersome. It has grown to the point where it was
taking about 5.5s to startup a shell! This is ok if I am going to spend
some time in here for awhile and do some work that benefits from all of the
autocompletions, plugins, and shortcuts that oh-my-zsh brings. But to only
jump in to run a handful of commands is infuriating.
📑 My Setup # [1]
I believe the real issue is io speed on wsl. I have some remote servers with
similar configs that are 10x faster or more, loading in 100s of milliseconds
rather than seconds. Sourcing all of the individual plugin files are just too
much for it.
💨 How Fast can it be # [2]
Quick side note: your zsh config is controled by your ~/.zshrc file. This
file can source other files, load plugins, or run literally anything.
Time the initial time
time zsh -c exit
Move your ~/.zshrc config file.
mv ~/.zshrc ~/.zshrc-back
Time the fastest startup p...
Keep Location List Closed
Vim’s (neovim in my case) location list can provide some very useful information while developing. Mine gives me information about linting and type checking errors with fairly little config. Generally, it sits nicely at the bottom of the screen and barely affects me. Other times, especially while zoomed way in during a presentation, it just gets in the way.
[1]
location list eats the screen
Location List eating up the screen while I am zoomed in and trying to live code
Toggling the location list # [2]
Through some google search I found the culprit was syntastic. It has an auto_loc_list feature. We can turn it off by setting
syntastic_auto_loc_list=0.
let syntastic_auto_loc_list=0
Keybindings # [3]
I want to keep the location list open automatically most of the time, but when I don’t want it to keep opening it’s generally detrimental. Trying to live code while the location list keeps taking up the whole screen is not cool.
First, create a function that will toggle both the lo...
SqlAlchemy Models
Make a connection # [1]
from sqlalchemy import create_engine
def get_engine():
return create_engine("sqlite:///mode_examples.sqlite")
Make a session # [2]
from sqlalchemy.orm import sessionmaker
def get_session():
con = get_engine()
Base.bind = con
Base.metadata.create_all()
Session = sessionmaker(bind=con)
session = Session()
return session
Make a Base Class # [3]
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
Base.metadata.bind = get_engine()
Make your First Model # [4]
class User(Base):
__tablename__ = "users"
username = Column('username', Text())
firstname = Column('firstname', Text())
lastname = Column('lastname', Text())
Make your own Base Class to inherit From # [5]
class MyBaseHelper:
def to_dict(self):
return {k: v for k, v in self.__dict__.items() if k[0] != "_"}
def update(self, **attrs):
for key, value in attrs.items():
if hasattr(self, key):
setattr(self, key, value)
Use the Custom Base Class # [6]
class User(Ba...
I came across jumpcutter [1] from carykh [2], and it’s packed with great features and ideas.
Automatically edits vidx. Explanation here: https://www.youtube.com/watch?v=DQ8orIurGxw
References:
[1]: https://github.com/carykh/jumpcutter
[2]: https://github.com/carykh
Check out great-expectations [1] and their project great_expectations [2].
Always know what to expect from your data.
References:
[1]: https://github.com/great-expectations
[2]: https://github.com/great-expectations/great_expectations
I’m impressed by flowy [1] from alyssaxuu [2].
The minimal javascript library to create flowcharts ✨
References:
[1]: https://github.com/alyssaxuu/flowy
[2]: https://github.com/alyssaxuu
I came across blocks [1] from blocks [2], and it’s packed with great features and ideas.
A JSX-based page builder for creating beautiful websites without writing code
References:
[1]: https://github.com/blocks/blocks
[2]: https://github.com/blocks