Published

Packages # Click # Inputs # Click primarily takes two forms of inputs Options and arguments. I think of options as keyword argument and arguments as regular positional arguments. Option # typically aliased with a shorthand (’-v’, ‘–verbose’) **From the Docs To get the Python argument name, the chosen name is converted to lower case, up to two dashes are removed as the prefix, and other dashes are converted to underscores. Argument # positional required no help text supplied by click Yaspin # Click Help Colors # # Colorama # Colorama Example Click DidYouMean #

See all of my kedro related posts in [[ tag/kedro]]. #kedrotips # I am tweeting out most of these snippets as I add them, you can find them all here #kedrotips. 🗣 Heads up # Below are some quick snippets/notes for when using kedro to build data pipelines. So far I am just compiling snippets. Eventually I will create several posts on kedro. These are mostly things that I use In my everyday with kedro. Some are a bit more essoteric. Some are helpful when writing production code, some are useful more usefule for exploration. 📚 Catalog # CSVLocalDataSet # python yaml CSVHTTPDataSet # HDFLocalDataSet # HDFS3LocalDataSet # JSONLocalDataSet # ParquetLocalDataSet # PickleS3DataSet SQLTableDataSet SQLQueryDataSet TextLocalDataSet ExcelLocalDataSet ⏳ Loading Data # Simple Loading # list all datasets # Saving Data # 🔍 Finding data # simple keyword search see on #kedrotips multi keyword serch see on #kedrotips 🐒 monkey patch it _see on #kedrotips 🤙 YOLO # You Only Load Once simple more refined 🍷 r…

jmespath Tabnine Bulwark # |-|-| |github: | https://github.com/zaxr/bulwark | I definitely want to try this out with kedro. Bulwark is a package for convenient property-based testing of pandas dataframes, supported for Python 3.5+. Example #

Using pdb #

Pathlib is an amazing cross-platform path tool. Import # Create path object # Current Directory Users Home Directory module directory Others Let’s create a path relative to our current module. Check if files exist # Make Directories # rename files # List files # Glob Files # recursively Write #

Custom Exceptions #

query # Good for method chaining, i.e. adding more methods or filters without assigning a new variable. masking # general purpose, this is probably the most common method you see in training/examples isin # capable of including multiple strings to include contains # Good For partial matches MASKS # anything that we put inside of square brackets can be set as a variable then passed in. Operators # & - and ~ - not | - or AVAILABLE and NAME # AVAILABLE or NAME # AVAILABLE and not NAME #