Python Bytes

By Michael Kennedy and Brian Okken

Listen to a podcast, please open Podcast Republic app. Available on Google Play Store.

Category: Software How-To

Open in iTunes

Open RSS feed

Open Website

Rate for this podcast

Subscribers: 643
Reviews: 2

 Dec 13, 2018
One of the best Python podcasts I've come across.

 Nov 25, 2018
Great summary of exciting Python news and opportunities to learn, both for the advanced and beginner Python programmer.


Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

Episode Date
#159 Brian's PR is merged, the src will flow
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>Michael #1: <a href=""><strong>Final type</strong></a></p> <ul> <li>PEP 591 -- Adding a final qualifier to typing</li> <li>This PEP proposes a "final" qualifier to be added to the typing module---in the form of a final decorator and a Final type annotation---to serve three related purposes: <ul> <li>Declaring that a method should not be overridden</li> <li>Declaring that a class should not be subclassed</li> <li>Declaring that a variable or attribute should not be reassigned</li> </ul></li> <li>Some situations where a final class or method may be useful include: <ul> <li>A class wasn’t designed to be subclassed or a method wasn't designed to be overridden. Perhaps it would not work as expected, or be error-prone.</li> <li>Subclassing or overriding would make code harder to understand or maintain. For example, you may want to prevent unnecessarily tight coupling between base classes and subclasses.</li> <li>You want to retain the freedom to arbitrarily change the class implementation in the future, and these changes might break subclasses.</li> </ul></li> </ul> <pre><code> # Example for a class: from typing import final @final class Base: ... class Derived(Base): # Error: Cannot inherit from final class "Base" ... </code></pre> <p>And for a method:</p> <pre><code> class Base: @final def foo(self) -&gt; None: ... class Derived(Base): def foo(self) -&gt; None: # Error: Cannot override final attribute "foo" # (previously declared in base class "Base") ... </code></pre> <ul> <li>It seems to also mean <code>const</code></li> </ul> <pre><code> RATE: Final = 3000 class Base: DEFAULT_ID: Final = 0 RATE = 300 # Error: can't assign to final attribute Base.DEFAULT_ID = 1 # Error: can't override a final attribute </code></pre> <p><strong>Brian #2:</strong> <a href=""><strong>flit 2</strong></a></p> <p><strong>Michael #3:</strong> <a href=""><strong>Pint</strong></a></p> <ul> <li>via Andrew Simon</li> <li>Physical units and builtin unit conversion to everyday python numbers like floats. </li> <li>Receive inputs in different unit systems it can make life difficult to account for that in software. </li> <li>Pint handles the unit conversion automatically in a wide array of contexts – Can add 2 meters and 5 inches and get the correct result without any additional work. </li> <li>The integration with numpy and pandas are seamless, and it’s made my life so much simpler overall. </li> <li><a href="">Units and types of measurements</a></li> <li>Think you need this? How about the Mars Climate Orbiter <ul> <li><em>The MCO MIB has determined that the root cause for the loss of the MCO spacecraft was the failure to use metric units in the coding of a ground software file, “Small Forces,” used in trajectory models. Specifically, thruster performance data in English units instead of metric units was used in the software application code titled SM_FORCES (small forces).</em></li> </ul></li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>8 great pytest plugins</strong></a></p> <ul> <li>Jeff Triplett</li> </ul> <p><strong>Michael #5: <a href="">11 new web frameworks</a></strong></p> <ul> <li>via <a href="">LuisCarlos Contreras</a></li> </ul> <ol> <li><strong>Sanic</strong> [flask like] - a web server and web framework that’s written to go fast. It allows the usage of the async / await syntax added in Python 3.5</li> <li><strong>Starlette</strong> [flask like] - A lightweight <a href="">ASGI</a> framework which is ideal for building high performance <code>asyncio</code> services, designed to be used either as a complete framework, or as an ASGI toolkit.</li> <li><strong>Masonite</strong> - A developer centric Python web framework that strives for an actual batteries included developer tool with a lot of out of the box functionality. Craft CLI is the edge here.</li> <li><strong>FastAPI</strong> - A modern, high-performance, web framework for building APIs with Python 3.6+ based on standard Python type hints.</li> <li><strong>Responder</strong> - Based on Starlette, Responder’s primary concept is to bring the niceties that are brought forth from both Flask and Falcon and unify them into a single framework.</li> <li><strong>Molten</strong> - A minimal, extensible, fast and productive framework for building HTTP APIs with Python. Molten can automatically validate requests according to predefined schemas.</li> <li><strong>Japronto</strong> - A screaming-fast, scalable, asynchronous Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on <code>uvloop</code> and <code>picohttpparser</code>.</li> <li><strong>Klein</strong> [flask like] - A micro-framework for developing production-ready web services with Python. It is ‘micro’ in that it has an incredibly small API similar to Bottle and Flask.</li> <li><strong>Quart</strong> [flask like]- A Python ASGI web microframework. It is intended to provide the easiest way to use asyncio functionality in a web context, especially with existing Flask apps.</li> <li><strong>BlackSheep</strong> - An asynchronous web framework to build event based, non-blocking Python web applications. It is inspired by Flask and ASP.NET Core. BlackSheep supports automatic binding of values for request handlers, by type annotation or by conventions.</li> <li><strong>Cyclone</strong> - A web server framework that implements the Tornado API as a Twisted protocol. The idea is to bridge Tornado’s elegant and straightforward API to Twisted’s Event-Loop, enabling a vast number of supported protocols.</li> </ol> <p><strong>Brian #6:</strong> <a href=""><strong>Raise Better Exceptions in Python</strong></a></p> <p>Extras</p> <p>Michael: </p> <ul> <li><a href="">Naming venvs</a> <code>--prompt</code> </li> <li>Another new course coming soon: <strong><a href="">Python for decision makers and business leaders</a></strong></li> <li>Some random interview over at Real Python: <a href="">Python Community Interview With Brian Okken</a></li> </ul> <p>Joke</p> <ul> <li>via Daniel Pope </li> <li>What's a tractor's <em>least</em> favorite programming language? <strong>Rust</strong>.</li> </ul>
Dec 03, 2019
#158 There's a bounty on your open-source bugs!
<p>This episode is sponsored by DigitalOcean - <a href=""><strong></strong></a></p> <p><strong>Brian</strong> <strong>#1:</strong> <a href=""><strong>Python already replaced Excel in banking</strong></a></p> <ul> <li>“If you wanted to prove your mettle as an <a href="">entry-level banker</a> or trader it used to be the case that you had to know all about financial modeling in Excel. Not any more. These days it's all about <a href="">Python</a>, especially on the trading floor. </li> <li>"Python already replaced Excel," said Matthew Hampson, deputy chief digital officer at Nomura, speaking at last Friday's Quant Conference in London. "You can already walk across the trading floor and see people writing Python will become much more common in the next three to four years."</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>GitHub launches 'Security Lab' to help secure open source ecosystem</strong></a></p> <ul> <li>At the GitHub Universe developer conference, GitHub announced the launch of a new community program called <a href="">Security Lab</a></li> <li>GitHub says Security Lab founding members have found, reported, and helped fix more than 100 security flaws already.</li> <li>Other organizations, as well as individual security researchers, can also join. A bug bounty program with rewards <a href="">of up to $3,000</a> is also available, to compensate bug hunters for the time they put into searching for vulnerabilities in open source projects.</li> <li>Bug reports must contain a CodeQL query. <a href="">CodeQL</a> is a new open source tool that GitHub released today; a semantic code analysis engine that was designed to find different versions of the same vulnerability across vasts swaths of code.</li> <li>Starting today automated security updates are generally available and have been rolled out to every active repository with security alerts enabled.</li> <li>Once a security flaw is fixed, the project owner can publish the security, and GitHub will warn all upstream project owners who are using vulnerable versions of the original maintainer's code.</li> <li>But before publishing a security advisory, project owners can also request and receive a CVE number for their project's vulnerability directly from GitHub.</li> <li>And last, but not least, GitHub also updated Token Scanning, its in-house service that can scan users' projects for API keys and tokens that have been accidentally left inside their source code.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong> now has some test challenges</strong></a></p> <ul> <li>Uses <a href="">pytest</a>, <a href=""></a>, and <a href="">mutpy</a> (for mutation testing)</li> <li>Most other challenges have tests that validate the code you write.</li> <li>New challenges (3 so far) have you write the tests for existing code.</li> <li>Tests are evaluated with both and mutpy</li> <li>another mutation testing tool is <a href="">mutmut</a>, <a href="">written about earlier this year by Ned Badtchelder</a>.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>pyhttptest - a command-line tool for HTTP tests over RESTful APIs</strong></a></p> <ul> <li>via <a href="">Florian Dahlitz</a></li> <li>A command-line tool for HTTP tests over RESTful APIs</li> <li>Tired of writing test scripts against your RESTFul APIs anytime? Describe an HTTP Requests test cases in a simplest and widely used format JSON within a file. Run one command and gain a summary report.</li> <li>Example</li> </ul> <pre><code> { "name": "TEST: List all users", "verb": "GET", "endpoint": "users", "host": "", "headers": { "Accept-Language": "en-US" }, "query_string": { "limit": 5 } } </code></pre> <p><strong>Brian #5:</strong> <a href=""><strong>xarray</strong></a></p> <ul> <li>suggested by Guido Imperiale</li> <li>xarray is a mature library that builds on top of numpy, pandas and dask to offer arrays that are <ul> <li>n-dimensional (numpy and dask do it, but pandas doesn't)</li> <li>self-described and indexed (pandas does it, but numpy and dask don't)</li> <li>out-of-memory, multi-threaded, and cloud-distributed (dask does it, but numpy and pandas don't).</li> </ul></li> <li>Additionally, xarray can semi-transparently swap numpy with other backends, such as <a href="">sparse</a> , while retaining the same API.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Animated SVG Terminals</strong></a></p> <ul> <li><a href="">Florian Dahlitz</a></li> <li><strong>termtosvg</strong> is a Unix terminal recorder <strong>written in Python</strong> that renders your command line sessions as standalone SVG animations.</li> <li>Features: <ul> <li>Produce lightweight and clean looking animations or still frames embeddable on a project page</li> <li>Custom color themes, terminal UI and animation controls via user-defined <a href="">SVG templates</a></li> <li>Rendering of recordings in asciicast format made with asciinema</li> </ul></li> <li>Examples: <a href=""></a></li> </ul> <p><strong>Extras</strong></p> <ul> <li><a href="">pytest 5.3.0</a> released, please read <a href="">changelog</a> if you use pytest, especially if you use it with CI and depend on <code>--junitxml</code>, as they have changed the format to a newer version.</li> </ul> <p>Michael: </p> <ul> <li><a href="">PyCon registration</a> is open (via Jacqueline Wilson)</li> <li>Facebook: <a href="">Microsoft's Visual Studio Code is now our default development platform</a></li> <li>Black friday at Talk Python Training!</li> <li>New course coming soon: Python for the .NET developer</li> </ul> <p><strong>Jokes</strong></p> <ul> <li><p>What do you get when you put root beer in a square glass? Beer.</p></li> <li><p><strong>Q:</strong> What do you call optimistic front-end developers?</p></li> <li><p><strong>A:</strong> Stack half-full developers.</p></li> <li><p>Also, I was going to tell a version control joke, but they are only funny if you <strong><em>git</em></strong> them.</p></li> </ul>
Nov 27, 2019
#157 Oh hai Pandas, hold my hand?
<p>This episode is sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Michael #1:</strong> <a href=""><strong>pydantic</strong></a></p> <ul> <li>via <a href="">Colin Sullivan</a></li> <li>Data validation and settings management using python type annotations.</li> <li>(We covered <a href="">Cerberus</a>, this is similar)</li> <li><em>pydantic</em> enforces type hints at runtime, and provides user friendly errors when data is invalid.</li> </ul> <pre><code> class User(pydantic.BaseModel): id: int name = 'John Doe' signup_ts: datetime = None friends: List[int] = [] external_data = { 'id': '123', 'signup_ts': '2019-06-01 12:22', 'friends': [1, 2, '3'] } user = User(**external_data) </code></pre> <ul> <li><code>id</code> is of type int; the annotation-only declaration tells <em>pydantic</em> that this field is required. Strings, bytes or floats will be coerced to ints if possible; otherwise an exception will be raised.</li> <li><code>name</code> is inferred as a string from the provided default; because it has a default, it is not required.</li> <li><code>signup_ts</code> is a datetime field which is not required (and takes the value <code>None</code> if it's not supplied).</li> <li>Why use it? <ul> <li>There's no new schema definition micro-language to learn.</li> <li>In <a href="">benchmarks</a> <em>pydantic</em> is faster than all other tested libraries.</li> <li>Use of <a href="">recursive</a> <a href=""><em>pydantic</em></a> <a href="">models</a>, <code>typing</code>'s <a href="">standard types</a> (e.g. <code>List</code>, <code>Tuple</code>, <code>Dict</code> etc.) and <a href="">validators</a> allow complex data schemas to be clearly and easily defined, validated, and parsed.</li> <li>As well as <code>BaseModel</code>, <em>pydantic</em> provides a <code>[dataclass](</code> decorator which creates (almost) vanilla python dataclasses with input data parsing and validation.</li> </ul></li> </ul> <p><strong>Brian #2:</strong> <strong> 5.0 beta 1 adds context support</strong></p> <ul> <li>Please try out the beta, even without trying contexts, as it helps Ned Batchelder to make sure it’s as backwards compatible as possible while still adding this super cool functionality. <ul> <li><a href="">Coverage 5.0 beta 1</a> announcement</li> <li>The <a href="">changes</a>.</li> <li><a href="">Measurement contexts</a> in depth.</li> </ul></li> <li>Trying out contexts with pytest and pytest-cov:</li> </ul> <pre><code> (venv) $ pip install coverage==5.0b1 (venv) $ pip install pytest-cov (venv) $ pytest --cov=foo --cov-context=test (venv) $ coverage html --show-contexts (venv) $ open htmlcov/index.html </code></pre> <ul> <li>results in coverage report that has little dropdowns on the right for lines that are covered, and what context they were covered.</li> <li>For the example above, with pytest-cov, it shows what test caused each line to be hit.</li> <li>Contexts can do way more than this. One example, split up different levels of tests, to see which lines are only hit by unit tests, indicating missing higher level tests, or the opposite.</li> <li>The stored db could also possibly be mined to see how much overlap there is between tests, and maybe help with higher level tools to predict the harm or benefit from removing some tests.</li> <li>I’m excited about the future, with contexts in place.</li> <li>Even if you ignore contexts, please go try out the beta ASAP to make sure your old use model still works.</li> </ul> <p><a href=""><strong>Michael #3: PSF is seeking developers for paid contract improving pip</strong></a></p> <ul> <li>via Brian Rutledge</li> <li>The <a href="">Python Software Foundation</a> <a href="">Packaging Working Group</a> is receiving funding to work on the design, implementation, and rollout of pip's next-generation dependency resolver.</li> <li>This project aims to complete the <a href="">design, implementation, and rollout</a> of <a href="">pip's next-generation dependency resolver</a>. </li> <li>Lower the barriers to installing Python software, empowering users to get a version of a package that works. </li> <li>It will also lower the barriers to distributing Python software, empowering developers to make their work available in an easily reusable form.</li> <li>Because of the size of the project, funding has been allocated to secure two contractors, a senior developer and an intermediate developer, to work on development, testing and building test infrastructure, code review, bug triage, and assisting in the rollout of necessary features.</li> <li>Total pay: Stage 1: $116,375, Stage 2: $103,700 </li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>dovpanda</strong></a> <strong>- Directions OVer PANDAs</strong></p> <ul> <li>Dean Langsam</li> <li>“Directions are hints and tips for using pandas in an analysis environment. dovpanda is an overlay for working with pandas in an analysis environment.</li> <li>"If you think your task is common enough, it probably is, and Pandas probably has a built-in solution. dovpanda is an overlay module that tries to understand what you are trying to do with your data, and help you find easier ways to write your code.”</li> <li>“The main usage of <code>dovpanda</code> is its hints mechanism, which is very easy and works out-of-the-box. Just import it after you import pandas, whether inside a notebook or in a console.”</li> <li>It’s like training wheels for pandas to help you get the most out of pandas and learn while you are doing your work. Very cool.</li> </ul> <p><strong>Michael #5:</strong> <a href=""><strong>removestar</strong></a> </p> <ul> <li>via PyCoders newsletter</li> <li>Tool to automatically replace 'import *' in Python files with explicit imports</li> <li>Report only mode and modify in place mode.</li> </ul> <p><strong>Brian #6:</strong> <a href=""><strong>pytest-quarantine</strong></a> <strong>: Save the list of failing tests, so that they can be automatically marked as expected failures on future test runs.</strong></p> <ul> <li>Brian Rutlage</li> <li>Really nice email from Brian: </li> <li>&gt;"Hi Brian! We've met a couple times at PyCon in Cleveland. Thanks for your podcasts, and your book. I've gone from being a complete pytest newbie, to helping my company adopt it, to writing a plugin. The plugin was something I developed at work, and they let me open-source it. I wanted to share it with you as a way of saying "thank you", and because you seem to be a bit of connoisseur of pytest plugins. ;)"</li> <li>Here it is: <a href=""></a>”</li> <li>pytest has a cool feature called xfail, to allow you to mark tests you know fail.</li> <li>pytest-quarantine allows you to run your suite and generate a file of all failures, then use that to mark the xfails. <ul> <li>Then you or your team can chip away at these failures until you get rid of them. </li> <li>But in the meantime, your suite can still be useful for finding new failures.</li> <li>And, the use of an external file to mark failures makes it so you don’t have to edit your test files to mark the tests that are xfail.</li> </ul></li> </ul> <p>Extras:</p> <p>MK: <a href="">Our infrastructure is fully carbon neutral</a>!</p> <p>Joke:</p> <p>A cop pulls Dr. Heisenberg over for speeding. The officer asks, "Do you know how fast you were going?" Heisenberg pauses for a moment, then answers, "No, but I know where I am.” [1]</p> <ol> <li><a href="">S</a><a href=""><strong>ee Uncertainty principle</strong></a>, also called <strong>Heisenberg uncertainty principle</strong> or indeterminacy <strong>principle</strong>, statement, articulated (1927) by the German physicist Werner <strong>Heisenberg</strong>, that the position and the velocity of an object cannot both be measured exactly, at the same time, even in theory.</li> </ol>
Nov 20, 2019
#156 All the programming LOLs
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Special guests</strong>:</p> <ul> <li><strong><a href="">Dan Bader</a></strong></li> <li><a href=""><strong>Cecil Philip</strong></a></li> </ul> <p><strong>Dan #1: Why You Should Use</strong> <code>python -m pip</code> </p> <ul> <li><a href=""></a></li> </ul> <p><strong>Cecil #2:</strong> <a href=""><strong>Visual Studio Online: Web-Based IDE &amp; Collaborative Code Editor</strong></a> </p> <ul> <li><a href=""></a></li> </ul> <p><strong>Michael #3</strong>: <a href=";correlation_id=0c49d598-6d88-4bcb-8ec8-06df91216732&amp;ref=email_digest&amp;ref_campaign=email_digest&amp;ref_source=email&amp;utm_content=post_body&amp;utm_medium=digest&amp;utm_name=top_posts&amp;utm_source=email&amp;utm_term=day&amp;%243p=e_as&amp;;_branch_match_id=630471982955108833"><strong>Python Adopts a 12-month Release Cycle</strong></a></p> <ul> <li>The <a href="">long discussion</a> on changing the Python project's release cadence has come to a conclusion: the project will now be releasing new versions on an annual basis.</li> <li>Described in PEP 602</li> <li>The steering council thinks that having a consistent schedule every year when</li> <li>we hit beta, RC, and final it will help the community: <ul> <li>Know when to start testing the beta to provide feedback</li> <li>Known when the expect the RC so the community can prepare their projects for the final release</li> <li>Know when the final release will occur to coordinate their own releases (if necessary) when the final release of Python occurs</li> <li>Allow core developers to more easily plan their work to make sure work lands in the release they are targeting</li> <li>Make sure that core developers and the community have a shorter amount of time to wait for new features to be released</li> </ul></li> </ul> <p><strong>Dan #4:</strong> <a href=""><strong>Black 19.10b0 Released</strong></a> — stable release coming soon</p> <ul> <li><a href=""></a></li> <li>“Black Friday” release date? <a href=""></a></li> <li>Playground: <a href=""></a></li> </ul> <p><strong>Cecil 5</strong>: <a href=""><strong>Navigating code on GitHub</strong></a></p> <ul> <li>Example: <a href=""></a></li> </ul> <p><strong>Michael #6</strong>: <a href=""><strong>lolcommits: selfies for software developers</strong></a>.</p> <ul> <li><strong>lolcommits</strong> takes a snapshot with your webcam every time you git commit code, and archives a lolcat style image with it. <strong>git blame</strong> has never been so much fun.</li> <li>Infinite uses: Animate your progress through a project and watch as you age. See what you looked like when you broke the build. Keep a joint lolrepository for your entire company.</li> <li>Plugins: Lolcommits allows a growing list of <a href="">plugins</a> to perform additional work on your lolcommit image after capturing.</li> <li>Animate: Configure lolcommits to generate an <a href="">animated GIF</a> with each commit for extra lulz!</li> </ul> <p>Extras:</p> <p>Dan:</p> <ul> <li>Article &amp; Course on Python 3.8 <ul> <li><a href=""></a></li> <li><a href=""></a></li> </ul></li> </ul> <p>Cecil:</p> <ul> <li><a href="">Twitch learning Python channel</a></li> </ul> <p>Michael:</p> <ul> <li>New Anvil course, free one - <a href=""></a></li> <li>PSF yearly survey is out: <a href=""></a></li> </ul> <p>Joke:</p> <p><a href="">LOLCODE</a></p>
Nov 15, 2019
#155 Guido van Rossum retires
<p>Sponsored by Datadog: <a href=""><strong></strong></a> </p> <p><strong>Michael #1:</strong> <a href=""><strong>Guido retires</strong></a></p> <ul> <li>Guido van Rossum has left DropBox and retired (<a href="">post</a>)</li> <li>Let’s all take a moment to say thank you.</li> <li>I wonder what will come next in terms of creative projects</li> <li>Some comments from community members (see <a href="">Twitter thread</a>)</li> </ul> <p><strong>Brian #2:</strong> <a href=""><strong>SeleniumBase</strong></a></p> <ul> <li>Automated UI Testing with <a href="">Selenium WebDriver</a> and <a href="">pytest</a>.</li> <li>Very expressive and intuitive automation library built on top of Selenium WebDriver. <a href="">method overview</a></li> <li>very readable (this is a workflow test, but still, quite readable): from seleniumbase import BaseCase</li> </ul> <pre><code> class MyTestClass(BaseCase): def test_basic(self):"") self.assert_title("xkcd: Python") self.assert_element('img[alt="Python"]')'a[rel="license"]') self.assert_text("free to copy and reuse") self.go_back()"link=About") self.assert_text("", "h2")"") self.update_text("", "xkcd book\n") self.assert_exact_text("xkcd: volume 0", "h3") </code></pre> <ul> <li>includes plugins for including screenshots in test results.</li> <li>supports major CI systems</li> <li>some cool features that I didn’t expect <ul> <li>user onboarding demos</li> <li>assisted QA (partially automated with manual questions)</li> <li>support for selenium grid</li> <li>logs of command line options, including headless</li> </ul></li> </ul> <p><strong>Michael #3:</strong> <a href=""><strong>Reimplementing a Solaris command in Python gained 17x performance improvement from C</strong></a></p> <ul> <li>Postmortem by Darren Moffat</li> <li>Is Python slow?</li> <li>A result of fixing a memory allocation issue in the <a href="">/usr/bin/listusers</a> command</li> <li>Decided to investigate if this ancient C code could be improved by conversion to Python.</li> <li>The C code was largely untouched since 1988 and was around 800 lines long, it was written in an era when the number of users was fairly small and probably existed in the local files /etc/passwd or a smallish NIS server.</li> <li>It turns out that the algorithm to implement the listusers is basically some simple set manipulation.</li> <li>Rewrite of listusers in <a href="">Python 3</a> turned out to be roughly a 10th of the number of lines of code</li> <li>But Python would be slower right ? Turns out it isn't and in fact for some of my datasets (that had over 100,000 users in them) it was 17 times faster.</li> <li>A few of the comments asked about the availability of the Python version. The listusers command in Oracle Solaris 11.4 SRU 9 and higher. Since we ship the /usr/bin/listusers file as the Python code you can see it by just viewing the file in an editor. Note though that is not open source and is covered by the Oracle Solaris licenses.</li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>20 useful Python tips and tricks you should know</strong></a></p> <ul> <li>I admit it, I’m capable of getting link-baited by the occasional listicle.</li> <li>Some great stuff, especially for people coming from other languages. <ul> <li>Chained assignment: <code>x = y = z = 2</code></li> <li>Chained comparison: <ul> <li><code>2 &lt; x &lt;= 8</code></li> <li><code>2 &lt; x &gt; 4</code></li> <li><code>0 &lt; x &lt; 4 &lt; y &lt; 16</code></li> </ul></li> </ul></li> <li>Multiple assignment: <code>x, y, z = 2, 4, 8</code></li> <li>More Advanced Multiple Assignment: <code>x, *y, z = 2, 4, 8, 16</code> <ul> <li>I’ve been using the * for unpacking a lot, especially with <code>*_</code></li> </ul></li> <li>Merge dictionaries: <code>z = {**x, **y}</code></li> <li>Join strings: <code>'_'.join(['u', 'v', 'w'])</code></li> <li>using <code>list(set(something))</code> to remove duplicates.</li> <li>aggregate elements. using zip to element-wise combine two or more iterables. </li> </ul> <pre><code> &gt;&gt;&gt; x = [1, 2, 3] &gt;&gt;&gt; y = ['a', 'b', 'c'] &gt;&gt;&gt; zip(x, y) [(1, 'a'), (2, 'b'), (3, 'c')] </code></pre> <ul> <li>and then some other weird stuff that I don’t find that useful.</li> </ul> <p><strong>Michael #5: <a href="">Complexity Waterfall</a></strong></p> <ul> <li>via <a href="">Ahrem Ahreff</a></li> <li>Heavy use of <a href="">wemake-python-styleguide</a></li> <li>Code smells!</li> <li>Use your refactoring tools and write tests.</li> <li>Automation enable an opportunity of “Continuous Refactoring” and “Architecture on Demand” development styles.</li> </ul> <p><strong>Brian #6:</strong> <a href=""><strong>Plynth</strong></a></p> <ul> <li>Plynth is a GUI framework for building cross-platform desktop applications with HTML, CSS and Python.</li> <li>Plynth has integrated the standard CPython implementation with Chromium's rendering engine. You can run your python scripts seamlessly with HTML/CSS instead of using Javascript with modules from pip</li> <li>Plynth uses Chromium/Electron for its rendering. With Plynth, every Javascript library turns into a Python module.</li> <li>Not open source. But free for individuals, including commercial use and education.</li> <li>A bunch of tutorial videos that are not difficult to follow, and not long, but… not really obvious code either. </li> <li>Python 3.6 and 3.7 development kits available</li> </ul> <p>Extras:</p> <p>Michael: </p> <ul> <li><a href=";px=Google-KCSAN-Sanitizer">Google Is Uncovering Hundreds Of Race Conditions Within The Linux Kernel</a></li> </ul> <p>Joke:</p> <ul> <li><strong>Q</strong>: What's a web developer's favorite tea?</li> <li><strong>A</strong>: URL gray</li> <li>via Aideen Barry</li> </ul>
Nov 06, 2019
#154 Code, frozen in carbon, on display for all
<p>Sponsored by Datadog: <a href=""><strong></strong></a> </p> <p><strong>Special guest: <a href="">Bob Belderbos</a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>Lesser Known Coding Fonts</strong></a></p> <ul> <li>Interesting examination of some coding fonts.</li> <li>Link to a great talk called <a href="">Cracking the Code</a>, by Jonathan David Ross, about coding fonts and Input.</li> </ul> <p>I’m trying out Input Mono right now, and quite like it.</p> <ul> <li>Fira code: <a href=""></a> </li> </ul> <p><strong>Bob #2:</strong> <a href=""><strong>Django Admin Handbook</strong></a></p> <ul> <li>As a Django developer knowing the admin is pretty important.</li> <li>Free ebook of 40 or so pages, you can consume it in one evening.</li> <li>There are a lot of good tricks, 3 I liked: <ul> <li><a href="">How to optimize queries in Django admin</a> (override <em>get_queryset</em>)</li> <li><a href="">How to export CSV from Django admin</a> (useful for data analysis in Jupyter for example) </li> <li><a href="">How to override save behaviour for Django admin</a> (used this to notify users upon publishing a new exercise on our platform)</li> </ul></li> <li>Some more cool ebooks on that site, e.g. <a href="">Tweetable #Python</a>.</li> </ul> <p><strong>Michael #3:</strong> <a href=""><strong>Your Guide to the CPython Source Code</strong></a></p> <ul> <li>Let’s talk about exploring the CPython code</li> <li>You’ll want to get the code: <code>git clone</code></li> <li>Compile the code (Anthony gives lots of steps for macOS, Windows, and Linux)</li> <li>Structure:</li> </ul> <pre><code> cpython/ │ ├── Doc ← Source for the documentation ├── Grammar ← The computer-readable language definition ├── Include ← The C header files ├── Lib ← Standard library modules written in Python ├── Mac ← macOS support files ├── Misc ← Miscellaneous files ├── Modules ← Standard Library Modules written in C ├── Objects ← Core types and the object model ├── Parser ← The Python parser source code ├── PC ← Windows build support files ├── PCbuild ← Windows build support files for older Windows versions ├── Programs ← Source code for the python executable and other binaries ├── Python ← The CPython interpreter source code └── Tools ← Standalone tools useful for building or extending Python </code></pre> <ul> <li>Some cool “hidden” goodies. For example, check out <code>Lib/concurrent/futures/</code>, it comes with a cool ascii diagram of the process.</li> <li>Lots more covered, that we don’t have time for <ul> <li>The Python Interpreter Process</li> <li>The CPython Compiler and Execution Loop</li> <li>Objects in CPython</li> <li>The CPython Standard Library</li> <li>Installing a custom version</li> </ul></li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>Six Django template tags not often used in tutorials</strong></a></p> <ul> <li>Here’s a few: <ul> <li><code>{% empty %}</code>, for use in <code>for</code> loops when the array is empty</li> <li><code>{% lorem \[count\] [method] [random] %}</code> for automatically filling with Lorem Ipsum text.</li> <li><code>{% verbatim %} … {% endverbatim %}</code>, stop the rendering engine from trying to parse it and replace stuff.</li> <li><a href=""></a></li> </ul></li> </ul> <p><strong>Bob #5:</strong> <strong><a href="">Beautiful code snippets with Carbon</a></strong></p> <ul> <li>Beautiful images, great for teaching Python / programming.</li> <li>Used by a lot of developer, <a href="">nice example I spotted today</a>.</li> <li>Supports typing and drag and drop, just generated <a href=";t=seti&amp;wt=none&amp;l=auto&amp;ds=true&amp;dsyoff=20px&amp;dsblur=68px&amp;wc=true&amp;wa=true&amp;pv=56px&amp;ph=56px&amp;ln=false&amp;fl=1&amp;fm=Hack&amp;fs=14px&amp;lh=133%2525&amp;si=false&amp;es=2x&amp;wm=false&amp;code=import%252520pytest%25250A%25250Afrom%252520scores%252520import%252520Player%25252C%252520calculate_score%25252C%252520get_winner%25250A%25250A%25250A%252540pytest.mark.parametrize(%252522arg%25252C%252520expected%252522%25252C%252520%25255B%25250A%252520%252520%252520%252520(%25255B1%25252C%2525203%25252C%2525202%25252C%2525205%25255D%25252C%2525205)%25252C%25250A%252520%252520%252520%252520(%25255B1%25252C%2525204%25252C%2525202%25252C%2525205%25255D%25252C%2525209)%25252C%25250A%252520%252520%252520%252520(%25255B1%25252C%2525201%25252C%2525201%25252C%2525201%25255D%25252C%2525200)%25252C%25250A%252520%252520%252520%252520(%25255B4%25252C%2525205%25252C%2525201%25252C%2525202%25255D%25252C%2525209)%25252C%25250A%252520%252520%252520%252520(%25255B6%25252C%2525206%25252C%2525205%25252C%2525205%25255D%25252C%25252022)%25252C%25250A%25255D)%25250Adef%252520test_calculate_score(arg%25252C%252520expected)%25253A%25250A%252520%252520%252520%252520assert%252520calculate_score(arg)%252520%25253D%25253D%252520expected%25250A%25250A%25250A%252540pytest.mark.parametrize(%252522arg%252522%25252C%252520%25255B%25250A%252520%252520%252520%252520%25255B4%25252C%2525205%25252C%2525206%25252C%252520%2527a%2527%25255D%25252C%25250A%252520%252520%252520%252520%25255B4%25252C%252520-5%25252C%252520-1%25252C%2525202%25255D%25252C%25250A%252520%252520%252520%252520%25255B4%25252C%2525207%25252C%2525206%25252C%2525202%25255D%25252C%25250A%25255D)%25250Adef%252520test_wrong_inputs(arg)%25253A%25250A%252520%252520%252520%252520with%252520pytest.raises(ValueError)%25253A%25250A%252520%252520%252520%252520%252520%252520%252520%252520calculate_score(arg)%25250A%25250A%25250Adef%252520test_winner_3_players()%25253A%25250A%252520%252520%252520%252520players%252520%25253D%252520%25255B%25250A%252520%252520%252520%252520%252520%252520Player(name%25253D%2527player%2525201%2527%25252C%252520scores%25253D%25255B1%25252C%2525203%25252C%2525202%25252C%2525205%25255D)%25252C%25250A%252520%252520%252520%252520%252520%252520Player(name%25253D%2527player%2525202%2527%25252C%252520scores%25253D%25255B1%25252C%2525201%25252C%2525201%25252C%2525201%25255D)%25252C%25250A%252520%252520%252520%252520%252520%252520Player(name%25253D%2527player%2525203%2527%25252C%252520scores%25253D%25255B4%25252C%2525205%25252C%2525201%25252C%2525202%25255D)%25252C%252520%252520%252523%252520max%2525209%25250A%252520%252520%252520%252520%25255D%25250A%252520%252520%252520%252520assert%252520get_winner(players)%252520%25253D%25253D%252520players%25255B-1%25255D%25250A%25250A%25250Adef%252520test_winner_shorter_score_len_raises_exception()%25253A%25250A%252520%252520%252520%252520players%252520%25253D%252520%25255B%25250A%252520%252520%252520%252520%252520%252520Player(name%25253D%2527player%2525201%2527%25252C%252520scores%25253D%25255B4%25252C%2525203%25252C%2525205%25252C%2525205%25255D)%25252C%25250A%252520%252520%252520%252520%252520%252520Player(name%25253D%2527player%2525202%2527%25252C%252520scores%25253D%25255B4%25252C%2525204%25252C%2525206%25255D)%25252C%252520%252520%252523%252520lacks%252520one%252520score%25250A%252520%252520%252520%252520%252520%252520Player(name%25253D%2527player%2525203%2527%25252C%252520scores%25253D%25255B4%25252C%2525205%25252C%2525206%25252C%2525206%25255D)%25252C%25250A%252520%252520%252520%252520%25255D%25250A%252520%252520%252520%252520with%252520pytest.raises(ValueError)%25253A%25250A%252520%252520%252520%252520%252520%252520%252520%252520get_winner(players)%25250A%25250A%25250Adef%252520test_winner_longer_score_len_raises_exception()%25253A%25250A%252520%252520%252520%252520players%252520%25253D%252520%25255B%25250A%252520%252520%252520%252520%252520%252520Player(name%25253D%2527player%2525201%2527%25252C%252520scores%25253D%25255B4%25252C%2525203%25252C%2525205%25252C%2525205%25252C%2525204%25255D)%25252C%25250A%252520%252520%252520%252520%252520%252520Player(name%25253D%2527player%2525202%2527%25252C%252520scores%25253D%25255B4%25252C%2525204%25252C%2525206%25252C%2525206%25252C%2525203%25252C%2525202%25255D)%25252C%252520%252520%252523%2525201%252520more%25250A%252520%252520%252520%252520%252520%252520Player(name%25253D%2527player%2525203%2527%25252C%252520scores%25253D%25255B4%25252C%2525205%25252C%2525206%25252C%2525206%25252C%2525205%25255D)%25252C%25250A%252520%252520%252520%252520%25255D%25250A%252520%252520%252520%252520with%252520pytest.raises(ValueError)%25253A%25250A%252520%252520%252520%252520%252520%252520%252520%252520get_winner(players)%25250A">this link</a> by dropping a test module onto the canvas!</li> <li>Great to expand Twitter char limit (we use it to <a href="">generate Python Tip images</a>).</li> <li>Follow the project <a href="">here</a>, seems they now integrate with Github.</li> </ul> <p><strong>Michael #6:</strong> <a href="http://Researchers find bug in Python script may have affected hundreds of studies"><strong>Researchers find bug in Python script may have affected hundreds of studies</strong></a></p> <ul> <li>More info via Mike Driscoll at <a href=""><strong>Thousands of Scientific Papers May be Invalid Due to Misunderstanding Python</strong></a></li> <li>In a paper published October 8, researchers at the University of Hawaii found that a programming error in a set of Python scripts commonly used for computational analysis of chemistry data returned varying results based on which operating system they were run on.</li> <li>Scientists did not understand that Python’s glob.glob() does not return sorted results</li> <li>Throwing doubt on the results of more than 150 published chemistry studies.</li> <li>the researcher were trying to analyze results from an experiment involving cyanobacteria discovered significant variations in results run against the same nuclear magnetic resonance spectroscopy (NMR) data.</li> <li>The scripts, called the "Willoughby-Hoye" scripts after their creators, were found to return correct results on macOS Mavericks and Windows 10. But on macOS Mojave and Ubuntu, the results were off by nearly a full percent.</li> <li>The module depends on the operating system for the order in which the files are returned. And the results of the scripts' calculations are affected by the order in which the files are processed.</li> <li>The fix: A simple <code>list.sort()</code>!</li> <li>Williams said he hopes the paper will get scientists to pay more attention to the computational side of experiments in the future.</li> </ul> <p>Extras:</p> <ul> <li>Nov 5 is the next <a href="">Python PDX West</a></li> <li><a href="">Using Big Tech Tools</a></li> </ul> <p>Working on: <a href="">PyBites platform</a>: added flake8/ black code formatting, UI enhancements. </p> <p>Michael:</p> <ul> <li><a href="">Bezos DDoS'd: Amazon Web Services' DNS systems knackered by hours-long cyber-attack</a></li> <li>PyPI Just Crossed the 200,000 Packages Threshold! (via RP)</li> <li>XKCD Date — via André Jaenisch, Enter <a href=""></a> and learn, that it was published on 24th Sep 2014.</li> </ul> <p>Joke:</p> <ul> <li><strong>Q:</strong> What did the Network Administrator say when they caught a nasty virus?</li> <li><strong>A:</strong> It hurts when IP</li> </ul>
Oct 29, 2019
#153 Auto format my Python please!
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>Michael #1: <a href=""><strong>Building a Python C Extension Module</strong></a></p> <ul> <li>Tutorial, learn to use the Python API to write Python C extension modules.</li> <li>And <ul> <li>Invoke C functions from within Python</li> <li>Pass arguments from Python to C and parse them accordingly</li> <li>Raise exceptions from C code and create custom Python exceptions in C</li> <li>Define global constants in C and make them accessible in Python</li> <li>Test, package, and distribute your Python C extension module</li> </ul></li> <li>Extending Your Python Program <ul> <li>there may be other lesser-used system calls that are only accessible through C</li> </ul></li> <li>Steps: Writing a Python Interface in C <ul> <li>Figure out the arguments (e.g. <code>int fputs(const char *, FILE *)</code> )</li> <li>Implement in C:</li> </ul></li> </ul> <pre><code> #include Python.h static PyObject *method_fputs(PyObject *self, PyObject *args) { char *str, *filename = NULL; int bytes_copied = -1; /* Parse arguments */ if(!PyArg_ParseTuple(args, "ss", &amp;str, &amp;filename)) { return NULL; } FILE *fp = fopen(filename, "w"); bytes_copied = fputs(str, fp); fclose(fp); return PyLong_FromLong(bytes_copied); } </code></pre> <ul> <li>In line 2, you declare the argument types you wish to receive from your Python code</li> <li>line 6, then you’ll see that <code>PyArg_ParseTuple()</code> copies into the char*’s</li> <li>Write regular C code (fopen, fputs)</li> <li>Return: PyLong_FromLong() creates a PyLongObject, which represents an integer object in Python. </li> <li>a few extra functions that are necessary <ul> <li>write definitions of your module and the methods it contains</li> <li>Before you can import your new module, you first need to build it. You can do this by using the Python package <code>distutils</code>.</li> </ul></li> </ul> <p><strong>Brian #2:</strong> <a href=""><strong>What’s New in Python 3.8 -</strong></a></p> <p>We’ve already talked about the big hitters:</p> <ul> <li>assignment expressions, (the walrus operator)</li> <li>positional only parameters, (the / in the param list)</li> <li>f-strings support = for self-documenting expressions and debugging</li> </ul> <p>There are a few more goodies I wanted to quickly mention:</p> <ul> <li>More async: <code>python -m asyncio</code> launches a native async REPL</li> <li>More helpful warnings and messages when <ul> <li>using <code>is</code> and <code>is not</code> to compare strings and integers and other types intended to be compared with <code>==</code> and <code>!=</code></li> <li>Missing the comma in stuff like <code>[(1,2) (3,4)]</code>. Happens all the time with parametrized testing</li> </ul></li> <li>you can do iterable unpacking in a yield or return statement <ul> <li><code>x = (1, 2, 3)</code></li> <li><code>a, *b = x</code></li> <li><code>return a, *b</code> &lt;- this used to be a syntax error <ul> <li>you had to do <code>return (a, *b)</code></li> </ul></li> </ul></li> <li>New module <code>importlib.metadata</code> lets you access things like version numbers or dependent library required version numbers, and cool stuff like that. <ul> <li>quite a few more goodies. I run through all my favorites on <a href=""></a></li> </ul></li> </ul> <p>Michael #3: <a href=""><strong>UK National Cyber Security Centre (NCSC) is warning developers of the risks of sticking with Python 2.7, particularly for library writers</strong></a></p> <ul> <li>NCSC likens companies continuing to use Python 2 past its EOL to tempting another WannaCry or Equifax incident. <ul> <li><a href="">Equifax details</a>: a vulnerability, dubbed CVE-2017-5638, was discovered in Apache Struts, an open source development framework for creating enterprise Java applications that Equifax, along with thousands of other websites, uses…</li> </ul></li> <li>Quote: "If you're still using 2.x, it's time to port your code to Python 3," the NCSC said. "If you continue to use unsupported modules, you are risking the security of your organisation and data, as vulnerabilities will sooner or later appear which nobody is fixing."</li> <li>Moreover: "If you maintain a library that other developers depend on, you may be preventing them from updating to 3," the agency added. "By holding other developers back, you are indirectly and likely unintentionally increasing the security risks of others.”</li> <li>"If migrating your code base to Python 3 is not possible, another option is to pay a commercial company to support Python 2 for you," the NCSC said.</li> <li>NCSC: If you don't migrate, you should expect security incidents</li> <li><strong>Python's popularity makes updating code imperative:</strong> The reason the NCSC is warning companies about Python 2's impending EOL is because of the language's success.</li> </ul> <p>Brian #4: <a href="">Pythonic News</a></p> <ul> <li>Sebastian A. Steins</li> <li>“A Hacker News lookalike written in Python/Django”</li> <li>“ powering <a href=""></a>"</li> <li>Cool that it’s open source, and on github</li> <li>Was submitted to us by Sebastian, and a few others too, so there is excitement.</li> <li>It’s like 6 days old and has 153 stars on github, 4 contributors, 18 forks.</li> <li>Fun. </li> </ul> <p>Michael #5: <a href="">Deep Learning Workstations, Servers, Laptops, and GPU Cloud</a></p> <ul> <li>GPU-accelerated with TensorFlow, PyTorch, Keras, and more pre-installed. Just plug in and start training. Save up to 90% by moving off your current cloud and choosing Lambda.</li> <li>They offer: <ul> <li>TensorBook: GPU Laptop for $2,934</li> <li>Lambda Quad: 4x GPU Workstation for $21,108 (yikes!)</li> <li>All in: Lambda Hyperplane: 8x Tesla V100 Server, starting at $114,274</li> </ul></li> <li>But compare to: <ul> <li>AWS EC2: p3.8xlarge @ $12.24 per Hour =&gt; $8,935 / month</li> </ul></li> </ul> <p><strong>Brian #6: <a href="">Auto formatters for Python</a></strong></p> <ul> <li>A comparison of autopep8, yapf, and black</li> <li>Auto formatters are super helpful for teams. They shut down the unproductive arguments over style and make code reviews way more pleasant. People can focus on content, not being the style police.</li> <li>We love black. But it might be a bit over the top for some people. Here are a couple of other alternatives.</li> <li><a href="">autopep8</a> - mostly focuses on PEP8 <ul> <li>“autopep8 automatically formats Python code to conform to the PEP 8 style guide. It uses the pycodestyle utility to determine what parts of the code needs to be formatted. autopep8 is capable of fixing most of the formatting issues that can be reported by pycodestyle.”</li> </ul></li> <li><a href="">black</a> - does more <ul> <li>doesn’t have many options, but you can alter line length, can turn of string quote normalization, and you can limit or focus the files it sees.</li> <li>does a cool “check that the reformatted code still produces a valid AST that is equivalent to the original.” but you can turn that off with <code>--fast</code></li> </ul></li> <li><a href="">yapf</a> - way more customizable. <ul> <li>Great if you want to auto format to a custom style guide.</li> <li>“The ultimate goal is that the code YAPF produces is as good as the code that a programmer would write if they were following the style guide. It takes away some of the drudgery of maintaining your code.”</li> </ul></li> <li>Article is cool in that it shows some sample code and how it’s changed by the different formatters.</li> </ul> <p>Extras:</p> <p>Michael:</p> <ul> <li>New courses coming</li> <li>Financial Aid Launches for PyCon US 2020!</li> </ul> <p>Joke:</p> <ul> <li><p><a href="">American Py Song</a></p></li> <li><p>From Eric Nelson:</p> <ul> <li>Math joke. “i is as complex as it gets. jk.”</li> </ul></li> </ul>
Oct 23, 2019
#152 You have 35 million lines of Python 2, now what?
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Michael #1:</strong> <a href=""><strong>JPMorgan’s Athena Has 35 Million Lines of Python 2 Code, and Won’t Be Updated to Python 3 in Time</strong></a></p> <ul> <li>With 35 million lines of Python code, the Athena trading platform is at the core of JPMorgan's business operations. A late start to migrating to Python 3 could create a security risk.</li> <li>Athena platform is used internally at JPMorgan for pricing, trading, risk management, and analytics, with tools for data science and machine learning. </li> <li>This extensive feature set utilizes over 150,000 Python modules, over 500 open source packages, and 35 million lines of Python code contributed by over 1,500 developers, according to data presented by Misha Tselman, executive director at J.P. Morgan Chase in a talk at PyData 2017.</li> <li>And JPMorgan is going to miss the deadline</li> <li>Roadmap puts "most strategic components" compatible with Python 3 by the end of Q1 2020</li> <li>JPMorgan uses Continuous Delivery, with 10,000 to 15,000 production changes per week</li> <li>"If you maintain a library that other developers depend on," the post states, "you may be preventing them from updating to 3. By holding other developers back, you are indirectly and likely unintentionally increasing the security risks of others," adding that developers who do not publish code publicly should "consider your colleagues who may also be using your code internally."</li> </ul> <p><strong>Brian #2:</strong> <a href=""><strong>organize</strong></a></p> <ul> <li>suggested by <a href="">Ariel Barkan</a></li> <li>a Python based file management automation tool</li> <li>configuration is via a yml file</li> <li>command line tool to organize your file system</li> <li>examples: <ul> <li>move all of your screenshots off of your desktop into a screenshots folder</li> <li>move old incomplete downloads into trash</li> <li>remove empty files from certain folders</li> <li>organize receipts and invoices into date based folders</li> </ul></li> </ul> <p><strong>Michael #3:</strong> <a href=""><strong>PEP 589 – TypedDict: Type Hints for Dictionaries With a Fixed Set of Keys</strong></a></p> <ul> <li>Author: Jukka Lehtosalo</li> <li>Sponsor: Guido van Rossum</li> <li>Status: Accepted</li> <li>Version: 3.8</li> <li>PEP 484 defines the type Dict[K, V] for uniform dictionaries, where each value has the same type, and arbitrary key values are supported. </li> <li>It doesn't properly support the common pattern where the type of a dictionary value depends on the string value of the key.</li> <li>Core idea: Consider creating a type to validate an arbitrary JSON document with a fixed schema</li> <li>Proposed syntax:</li> </ul> <pre><code> from typing import TypedDict class Movie(TypedDict): name: str year: int movie: Movie = {'name': 'Blade Runner', 'year': 1982} </code></pre> <ul> <li>Operations on movie can be checked by a static type checker</li> </ul> <pre><code> movie['director'] = 'Ridley Scott' # Error: invalid key 'director' movie['year'] = '1982' # Error: invalid value type ("int" expected) </code></pre> <p><strong>Brian #4:</strong> <a href=""><strong>gazpacho</strong></a></p> <ul> <li>gazpacho is a web scraping library</li> <li>“It replaces requests and BeautifulSoup for most projects. “</li> <li>“gazpacho is small, simple, fast, and consistent.”</li> <li><a href="">example of using gazpacho</a> to scrape hockey data for fantasy sports.</li> <li>simple interface, short scripts, really beginner friendly</li> <li>retrieve with <code>get</code>, parse with <code>Soup</code>.</li> <li>I don’t think it will completely replace the other tools, but for simple get/parse/find operations, it may make for slimmer code.</li> <li>Note, I needed to update certificates to get this to work. see <a href="">this</a>.</li> </ul> <p><strong>Michael #5:</strong> <a href=""><strong>How pip install Works</strong></a></p> <ul> <li>via PyDist</li> <li>What happens when you run pip install [somepackage]?</li> <li>First pip needs to decide which distribution of the package to install. <ul> <li>This is more complex for Python than many other languages</li> </ul></li> <li>There are 7 different kinds of distributions, but the most common these days are source distributions and binary wheels. </li> <li>A binary wheel is a more complex archive format, which can contain compiled C extension code. </li> <li>Compiling, say, numpy from source takes a long time (~4 minutes on my desktop), and it is hard for package authors to ensure that their source code will compile on other people's machines.</li> <li>Most packages with C extensions will build multiple wheel distributions, and pip needs to decide which if any are suitable for your computer.</li> <li>To find the distributions available, pip requests[somepackage], which is a simple HTML page full of links, where the text of the link is the filename of the distribution.</li> <li>To select a distribution, pip first determines which distributions are compatible with your system and implementation of python. <ul> <li>binary wheels, it parses the filenames according to PEP 425, extracting the python implementation, application binary interface, and platform.</li> <li>All source distributions are assumed to be compatible, at least at this step in the process</li> </ul></li> <li>Once pip has a list of compatible distributions, it sorts them by version, chooses the most recent version, and then chooses the "best" distribution for that version</li> <li>It prefers binary wheels if there are any</li> <li>Determining the dependencies for this distribution is not simple either. </li> <li>For binary wheels, the dependencies are listed in a file called METADATA. But for source distributions the dependencies are effectively whatever gets installed when you execute their script with the install command. </li> <li>What happens though if one of the distributions pip finds violates the requirements of another? It ignores the requirement and installs idna anyway!</li> <li>Next pip has to actually build and install the package.</li> <li>it needs to determine which library directory to install the package in—the system's, the user's, or a virtualenvs? </li> <li>Controlled by sys.prefix, which in turn is controlled by pip's executable path and the PYTHONPATH and PYTHONHOME environment variables. </li> <li>Finally, it moves the wheel files into the appropriate library directory, and compiles the python source files into bytecode for faster execution.</li> <li>Now your package is installed!</li> </ul> <p><strong>Brian #6:</strong> <a href=""><strong>daily pandas tricks</strong></a></p> <ul> <li>Kevin Markham is sending out one pandas tip or trick per day via twitter.</li> <li>It’s been fun to watch and learn new bits.</li> <li>The link is a sampling of a bunch of them.</li> <li>Here’s just one example:</li> </ul> <pre><code> Need to rename all of your columns in the same way? Use a string method: Replace spaces with _: df.columns = df.columns.str.replace(' ', '_') Make lowercase &amp; remove trailing whitespace: df.columns = df.columns.str.lower().str.rstrip() </code></pre> <p>Extras</p> <p>Michael:</p> <ul> <li>Switched to <a href="">Adobe Audition</a></li> <li><a href="">Azure Databricks drops Python 2</a></li> <li><a href="">Better Jupyter in VS Code</a> </li> <li>macOS Catalina (so far so good)</li> </ul> <p>Jokes:</p> <ul> <li>via <a href="">Sarcastic Pharmacist</a></li> <li><a href="">Hard to distinguish hard from easy in programming</a></li> </ul>
Oct 15, 2019
#151 Certified! It works on my machine
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Python alternative to Docker</strong></a></p> <ul> <li>Matt Layman</li> <li>Using <a href="">Shiv</a>, from LinkedIn <ul> <li>Mentioned briefly in <a href="">episode 114</a></li> <li>Shiv uses zipapp, <a href="">PEP 441</a>.</li> <li>Execute code directly from a zip file.</li> <li>App code and dependencies can be bundled together.</li> <li>“Having one artifact eliminates the possibility of a bad interaction getting to your production system.”</li> <li>article includes an example of all the steps for packaging a Django app with Gunicorn.</li> <li>includes talking about deployment.</li> </ul></li> <li>Matt includes shoutouts to: <ul> <li>Platform as a Service providers</li> <li>Manual steps to do it all.</li> <li>Docker</li> </ul></li> <li>Compares the process against Docker and discusses when to choose one over the other.</li> <li>Also an interesting read: <a href="">Docker is in deep trouble</a></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>How to support open-source software and stay sane</strong></a></p> <ul> <li>via <a href=""><strong>Jason Thomas</strong></a> written by <strong>Anna Nowogrodzki</strong> </li> <li>Releasing lab-built open-source software often involves a mountain of unforeseen work for the developers.</li> <li>Article opens: “On 10 April, astrophysicists announced that they had captured the first ever image of a black hole. This was exhilarating news, but none of the giddy headlines mentioned that the image would have been impossible without open-source software.”</li> <li>The image was created using Matplotlib, a Python library for graphing data, as well as other components of the open-source Python ecosystem. <strong>Just five days later</strong>, the US National Science Foundation <strong>(NSF) rejected a grant proposal to support that ecosystem</strong>, saying that the software lacked sufficient impact.</li> <li>Open-source software is widely acknowledged as crucially important in science, yet it is funded non-sustainably.</li> <li>“It’s sort of the difference between having insurance and having a GoFundMe when their grandma goes to the hospital,” says Anne Carpenter</li> <li>Challenges <ul> <li>Scientists writing open-source software often lack formal training in software engineering.</li> <li>Yet poorly maintained software can waste time and effort, and hinder reproducibility.</li> </ul></li> <li>If your research group is planning to release open-source software, you can prepare for the support work</li> <li>Obsolescence isn’t bad, she adds: knowing when to stop supporting software is an important skill.</li> <li>However long your software will be used for, good software-engineering practices and documentation are essential.</li> <li>These include continuous integration systems (such as TravisCI), version control (Git) and unit testing.</li> <li>To facilitate maintenance, Varoquaux recommends focusing on code readability over peak performance.</li> </ul> <p><strong>Brian #3</strong>: <a href=""><strong>The Hippocratic License</strong></a></p> <ul> <li>Coraline Ada Ehmke</li> <li>Interesting idea to derive from MIT, but add restrictions.</li> <li>This license adds these restrictions: <ul> <li>“The software may not be used by individuals, corporations, governments, or other groups for systems or activities that actively and knowingly endanger, harm, or otherwise threaten the physical, mental, economic, or general well-being of individuals or groups in violation of the United Nations Universal Declaration of Human Rights”</li> </ul></li> <li>I could see others with different restrictions, or this but more.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>MATLAB vs Python: Why and How to Make the Switch</strong></a></p> <ul> <li>MATLAB® is widely known as a high-quality environment for any work that involves arrays, matrices, or linear algebra.</li> <li>I personally used it for wavelet-decomposition of real time eye measurements during cognitively intensive human workloads… That <em>toolbox</em> costs $2000 per user.</li> <li>Difference in philosophy: Closed, paid vs. open source.</li> <li>Since Python is available at no cost, a much broader audience can use the code you develop</li> <li>Also, there is <a href="">GNU Octave</a> is a free and open-source clone of MATLAB apparently</li> </ul> <p><strong>Brian #5</strong>: <a href=""><strong>PyperCard - Easy GUIs for All</strong></a></p> <ul> <li>Nicholas Tollervey</li> <li>Came up on <a href="">episode 143</a></li> <li>Also, <a href="">episode 89 of Test &amp; Code</a></li> <li>Really easy to quickly set up a GUI specified by a list of “Card” objects. (different from <a href="">cards project</a>)</li> <li>Simple examples are choose your own adventure type applications, where one button takes you to another card, and another button, a different card.</li> <li>However, the “next card” could be a Python function that can do anything, as long as it returns a string with the name of the next card.</li> <li>Lots of potential here, especially with input boxes, images, sound, and more.</li> <li>Super fun, but also might have business use.</li> </ul> <p><strong>Michae #6</strong>: <a href=""><strong>pynode</strong></a></p> <ul> <li>Article: <a href="">Bridging Node.js and Python with PyNode to Predict Home Prices</a></li> <li>Call python code from node.js <ul> <li>Define a Python method</li> <li>In node: require pynode: <code>const pynode = require('@fridgerator/pynode')</code></li> <li>Start an interpreter: <code>pynode.startInterpreter()</code></li> <li>Call the function</li> </ul></li> </ul> <pre><code>'add', 1, 2, (err, result) =&gt; { if (err) return console.log('error : ', err) result === 3 // true }) </code></pre> <p><strong>Jokes</strong> </p> <p><a href=""><strong>The "Works on My Machine" Certification Program, get certified!</strong></a></p>
Oct 10, 2019
#150 Winning the Python software interview
<p>Sponsored by Datadog: <a href=""><em></em></a></p> <p>Michael #1: <a href=""><strong>How to Stand Out in a Python Coding Interview</strong></a></p> <ul> <li>Real Python, by <a href="">James Timmins</a></li> <li>Are tech interviews broken? Well at least we can try to succeed at them anyway</li> <li>You’ve made it past the phone call with the recruiter, and now it’s time to show that you know how to solve problems with actual code…</li> <li>Interviews aren’t just about solving problems: they’re also about showing that you can write clean production code. This means that you have a deep knowledge of Python’s built-in functionality and libraries.</li> <li>Things to learn <ul> <li>Use <code>enumerate()</code> to iterate over both indices and values</li> <li>Debug problematic code with <code>breakpoint()</code></li> <li>Format strings effectively with f-strings</li> <li>Sort lists with custom arguments</li> <li>Use generators instead of list comprehensions to conserve memory</li> <li>Define default values when looking up dictionary keys</li> <li>Count hashable objects with the <code>collections.Counter</code> class</li> <li>Use the standard library to get lists of permutations and combinations</li> </ul></li> </ul> <p><strong>Brian #2:</strong> <a href=""><strong>The Python Software Foundation has updated its Code of Conduct</strong></a></p> <ul> <li>There’s now one code of conduct for PSF and PyCon US and other spaces sponsored by the PSF</li> <li>This includes some regional conferences, such as PyCascades, and some meetup groups, (ears perk up)</li> <li>The docs <ul> <li><a href="">Code of Conduct</a></li> <li><a href="">Enforcement Guidelines</a></li> <li><a href="">Reporting Guidelines</a></li> </ul></li> <li>Do we need to care? <ul> <li>all of us, yes. If there weren’t problems, we wouldn’t need these.</li> <li>attendees, yes. Know before you go.</li> <li>organizers, yes. Better to think about it ahead of time and have a plan than have to make up a strategy during an event if something happens.</li> <li>me, in particular, and Michael. Ugh. yes. our first meetup is next month. I’d like to be in line with the rest of Python. So, yep, we are going to have to talk about this and put something in place.</li> </ul></li> </ul> <p>Michael #3: <a href=""><strong>The Interview Study Guide For Software Engineers</strong></a></p> <ul> <li>A checklist on my last round of interviews that covers many of the popular topics.</li> <li>Warm Up With The Classics <ol> <li><a href="">Fizz Buzz</a></li> <li><a href="">560. Subarray Sum Equals K</a></li> <li><a href=";playlist_slugs%5B%5D=interview-preparation-kit&amp;playlist_slugs%5B%5D=arrays">Arrays: Left Rotation</a></li> <li><a href=";playlist_slugs%5B%5D=interview-preparation-kit&amp;playlist_slugs%5B%5D=strings">Strings: Making Anagrams</a></li> <li><a href="">Nth Fibonacci</a></li> </ol></li> <li>Many many videos on interview topics and ideas <ul> <li>Data Structures </li> <li>Algorithms </li> <li>Big O Notation</li> <li>Dynamic Programming</li> <li>String Manipulation</li> <li>System Design</li> <li>Operating Systems</li> <li>Threads</li> <li>Object Oriented</li> <li>Design Patterns</li> <li>SQL</li> </ul></li> <li>Fun conversation in the comments</li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>re-assert</strong></a> <strong>: “show where your regex match assertion failed”</strong></p> <ul> <li>Anthony Sotille</li> <li>“<code>re-assert</code> provides a helper class to make assertions of regexes simpler.”</li> <li>The <code>Matches</code> objects allows for useful pytest assertion messages</li> <li>In order to get my head around it, I looked at the test code: <ul> <li><a href=""></a></li> <li>and modified it to remove all of the <code>with pytest.raises(AssertionError)…</code> to actually get to see the errors and how to use it.</li> </ul></li> </ul> <pre><code> def test_match_old(): &gt; assert re.match('foo', 'fob') E AssertionError: assert None E + where None = [HTML_REMOVED]('foo', 'fob') E + where [HTML_REMOVED] = re.match AssertionError ____________ test_match_new ___________________ def test_match_new(): &gt; assert Matches('foo') == 'fob' E AssertionError: assert Matches('foo') ^ == 'fob' E -Matches('foo') E - # regex failed to match at: E - # E - #&gt; fob E - # ^ E +'fob' </code></pre> <p>Michael #5: <a href=""><strong>awesome-python-typing</strong></a></p> <ul> <li>Collection of awesome Python types, stubs, plugins, and tools to work with them.</li> <li>Taxonomy <ul> <li><a href="">Static type checkers</a></li> <li><a href="">Stub packages</a></li> <li><a href="">Tools</a></li> <li><a href="">Integrations</a></li> <li><a href="">Articles</a></li> <li><a href="">Communities</a></li> <li><a href="">Related</a></li> </ul></li> <li>Static type checkers: <a href="">mypy</a> - Optional static typing for Python 3 and 2 (PEP 484).</li> <li>Stub packages: <a href="">Typeshed</a> - Collection of library stubs for Python, with static types.</li> <li>Tools (super category): <a href="">pytest-mypy</a> - Mypy static type checker plugin for Pytest.</li> <li>Articles: <a href="">Typechecking Django and DRF</a> - Full tutorial about type-checking django.</li> </ul> <p><strong>Brian #6:</strong> <a href=""><strong>Developer Advocacy: Frequently Asked Questions</strong></a></p> <ul> <li>Dustin Ingram</li> <li>I know a handful of people who have this job title. What is it? </li> <li>disclaimer: Dustin is a DA at Google. Other companies might be different</li> <li>What is it? <ul> <li>“I help represent the Python community at [company]"</li> <li>“part of my job is to be deeply involved in the Python community.”</li> <li>working on projects that help Python, PyPI, packaging, etc.</li> <li>speaking at conferences</li> <li>talking to people. customers and non-customers</li> <li>talking to product teams</li> <li>being “user zero” for new products and features</li> <li>paying attention to places users might raise issues about products</li> <li>working in open source</li> <li>creating content for Python devs</li> <li>being involved in the community as a company rep</li> <li>representing Python in the company</li> <li>coordinating with other DAs</li> </ul></li> <li><p>Work/life?</p> <ul> <li>Not all DAs travel all the time. that was my main question.</li> </ul></li> <li><p>Talk Python episode: <a href="">War Stories of the Developer Evangelists</a></p></li> </ul> <p>Extras:</p> <ul> <li><a href=""></a></li> </ul> <p>Michael:</p> <ul> <li><a href="">requests moves to PSF</a></li> </ul> <p>Joke:</p> <p>via <a href=""></a></p> <p><a href="">Web Dev Merit Badges</a></p>
Oct 05, 2019
#149 Python's small object allocator and other memory features
<p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Dropbox: Our journey to type checking 4 million lines of Python</strong></a></p> <ul> <li>Continuing saga, but this is a cool write up.</li> <li>Benefits <ul> <li>“Experience tells us that understanding code becomes the key to maintaining developer productivity. Without type annotations, basic reasoning such as figuring out the valid arguments to a function, or the possible return value types, becomes a hard problem. Here are typical questions that are often tricky to answer without type annotations: <ul> <li>Can this function return None?</li> <li>What is this items argument supposed to be?</li> <li>What is the type of the id attribute: is it int, str, or perhaps some custom type?</li> <li>Does this argument need to be a list, or can I give a tuple or a set?”</li> </ul></li> <li>Type checker will find many subtle bugs.</li> <li>Refactoring is easier.</li> <li>Running type checking is faster than running large suites of unit tests, so feedback can be faster.</li> <li>Typing helps IDEs with better completion, static error checking, and more.</li> </ul></li> <li>Long story, but really cool learnings of how and why to tackle adding type hints to a large project with many developers.</li> <li>Conclusion. mypy is great now, because DropBox needed it to be.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Setting Up a Flask Application in Visual Studio Code</strong></a></p> <ul> <li>Video, but also as <a href="">a post</a></li> <li>Follow on to the same in PyCharm: <ul> <li><a href="">video</a> and <a href="">post</a></li> </ul></li> <li>Steps outside VS Code <ul> <li>Clone repo</li> <li>Create a virtual env (via venv)</li> <li>Install requirements (via requirements.txt)</li> <li>Setup flask app ENV variable</li> <li>flask deploy ← custom command for DB</li> </ul></li> <li>VS Code <ul> <li>Open the folder where the repo and venv live</li> <li>Open any Python file to trigger the Python subsystem</li> <li>Ensure the correct VENV is selected (bottom left)</li> <li>Open the debugger tab, add config, pick Flask, choose your file</li> <li>Debug menu, start without debugging (or with)</li> </ul></li> <li>Adding tests via VS Code <ul> <li>Open command pallet (CMD SHIFT P), Python: Discover Tests, select framework, select directory of tests, file pattern, new tests bottle on the right bar</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Multiprocessing vs. Threading in Python: What Every Data Scientist Needs to Know</strong></a></p> <ul> <li>How data scientists can go about choosing between the multiprocessing and threading and which factors should be kept in mind while doing so.</li> <li>Does not consider async, but still some great info.</li> <li>Overview of both concepts in general and some of the pitfalls of parallel computing.</li> <li>The specifics in Python, with the GIL</li> <li>Use threads for waiting on IO or waiting on users.</li> <li>Use multiprocessing for CPU intensive work.</li> <li>The surprising bit for me was the benchmarks <ul> <li>Using something speeds up the code. That’s obvious.</li> <li>The difference between the two isn’t as great as I would have expected.</li> </ul></li> <li>A discussion of merits and benefits of both.</li> <li>And from the perspective of data science.</li> <li>A few more examples, with code, included.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>ORM - async ORM</strong></a></p> <ul> <li>And <a href=""></a></li> <li>The orm package is an async ORM for Python, with support for Postgres, MySQL, and SQLite.</li> <li><a href="">SQLAlchemy core</a> for query building.</li> <li><a href="">databases</a> for cross-database async support.</li> <li><a href="">typesystem</a> for data validation.</li> <li>Because ORM is built on SQLAlchemy core, you can use Alembic to provide database migrations.</li> <li>Need to be pretty async savy</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Getting Started with APIs</strong></a></p> <ul> <li> post</li> <li>Conceptual introduction of web APIs</li> <li>Discussion of GET status codes, including a nice list with descriptions. <ul> <li>examples: <ul> <li><code>301</code>: The server is redirecting you to a different endpoint. This can happen when a company switches domain names, or an endpoint name is changed.</li> <li><code>400</code>: The server thinks you made a bad request. This can happen when you don’t send along the right data, among other things.</li> </ul></li> </ul></li> <li>endpoints</li> <li>endpoints that take query parameters</li> <li>JSON data</li> <li>Examples in Python for using: <ul> <li><code>requests</code> to query endpoints.</li> <li><code>json</code> to load and dump JSON data.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Memory management in Python</strong></a></p> <ul> <li>This article describes memory management in Python 3.6.</li> <li>Everything in Python is an object. Some objects can hold other objects, such as lists, tuples, dicts, classes, etc.</li> <li>such an approach requires a lot of small memory allocations</li> <li>To speed-up memory operations and reduce fragmentation Python uses a special manager on top of the general-purpose allocator, called PyMalloc.</li> <li>Layered managers <ul> <li>RAM</li> <li>OS VMM</li> <li>C-malloc</li> <li>PyMem</li> <li>Python Object allocator</li> <li>Object memory</li> </ul></li> <li>Three levels of organization <ul> <li>To reduce overhead for small objects (less than 512 bytes) Python sub-allocates big blocks of memory.</li> <li>Larger objects are routed to standard C allocator.</li> <li>three levels of abstraction — <code>arena</code>, <code>pool</code>, and <code>block</code>.</li> <li><strong>Block</strong> is a chunk of memory of a certain size. Each block can keep only one Python object of a fixed size. The size of the block can vary from 8 to 512 bytes and must be a multiple of eight</li> <li>A collection of blocks of the same size is called a pool. Normally, the size of the <strong>pool</strong> is equal to the size of a <a href="">memory page</a>, i.e., 4Kb.</li> <li>The <strong>arena</strong> is a chunk of 256kB memory allocated on the heap, which provides memory for 64 pools.</li> </ul></li> <li>Python's small object manager rarely returns memory back to the Operating System.</li> <li>An arena gets fully released If and only if all the pools in it are empty.</li> </ul> <p><strong>Extras</strong></p> <p>Brian:</p> <ul> <li>Tuesday, Oct 6, <a href="">Python PDX West</a>, </li> <li>Thursday, Sept 26, I’ll be speaking at <a href="">PDX Python</a>, downtown.</li> <li>Both events, mostly, I’ll be working on new programming jokes unless I come up with something better. :)</li> </ul> <p>Michael:</p> <ul> <li><a href="">Gitbook</a></li> <li><a href="">Call for Proposals for PyCon 2020 Is Open</a></li> </ul> <p><strong>Jokes:</strong> A few I liked from the dad joke list.</p> <ul> <li>What do you call a 3.14 foot long snake? A π-thon</li> <li>What if it’s 3.14 inches, instead of feet? A μ-π-thon</li> <li>Why doesn't Hollywood make more Big Data movies? NoSQL.</li> <li>Why didn't the <code>div</code> get invited to the dinner party? Because it had no class.</li> </ul>
Sep 25, 2019
#148 The ASGI revolution is upon us!
<p>Sponsored by <strong>DigitalOcean:</strong> <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Annual Release Cycle for Python</strong></a> <strong>- PEP 602</strong></p> <ul> <li><a href="">Under discussion</a></li> <li>Annual release cadence</li> <li>Seventeen months to develop a major version <ul> <li>5 months unversioned, 7 months alpha releases when new features and fixes come in, 4 months of betas with no new features, 1 month final RCs.</li> </ul></li> <li>One year of full support, four more years of security fixes.</li> <li>Rationale/Goals <ul> <li>smaller releases</li> <li>features and fixes to users sooner</li> <li>more gradual upgrade path</li> <li>predictable calendar releases that line up will with sprints and PyConUS</li> <li>explicit alpha phase</li> <li>decrease pressure and rush to get features into beta 1</li> </ul></li> <li>Risks <ul> <li>Increase concurrent supported versions from 4 to 5.</li> <li>Test matrix increase for integrators and distributions.</li> </ul></li> <li>PEP includes rejected ideas like: <ul> <li>9 month cadence</li> <li>keep 18 month cadence</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>awesome-asgi</strong></a></p> <ul> <li>A curated list of awesome ASGI servers, frameworks, apps, libraries, and other resources </li> <li><a href="">ASGI</a> is a standard interface positioned as a spiritual successor to WSGI. It enables communication and interoperability across the whole Python async web stack: servers, applications, middleware, and individual components.</li> <li>Born in 2016 to power the Django Channels project, ASGI and its ecosystem have been expanding ever since, boosted by the arrival of projects such as Starlette and Uvicorn in 2018.</li> <li><em>Frameworks for building ASGI web applications.</em> <ul> <li><a href="">Bocadillo</a> - Fast, scalable and real-time capable web APIs for everyone. Powered by Starlette. Supports HTTP (incl. SSE) and WebSockets.</li> <li><a href="">Channels</a> - Asynchronous support for Django, and the original driving force behind the ASGI project. Supports HTTP and WebSockets with Django integration, and any protocol with ASGI-native code.</li> <li><a href="">FastAPI</a> - A modern, high-performance web framework for building APIs with Python 3.6+ based on standard Python type hints. Powered by Starlette and Pydantic. Supports HTTP and WebSockets.</li> <li><a href="">Quart</a> - A Python ASGI web microframework whose API is a superset of the Flask API. Supports HTTP (incl. SSE and HTTP/2 server push) and WebSockets.</li> <li><a href="">Responder</a> - A familiar HTTP Service Framework for Python, powered by Starlette. (ASGI 2.0 only, ed.)</li> <li><a href="">Starlette</a> - Starlette is a lightweight ASGI framework/toolkit, which is ideal for building high performance asyncio services. Supports HTTP and WebSockets.</li> </ul></li> </ul> <p><strong>Brian #3</strong>: <a href=""><strong>Jupyter meets the Earth</strong></a></p> <ul> <li>Lindsey Heagy &amp; Fernando Pérez</li> <li>“We are thrilled to announce that the NSF is funding our EarthCube proposal <em>“Jupyter meets the Earth: Enabling discovery in geoscience through interactive computing at scale”</em> “</li> <li>“This project provides our team with $2 Million in funding over 3 years as a part of the NSF <a href="">EarthCube</a> program. It also represents the first time federal funding is being allocated for the development of core Jupyter infrastructure.”</li> <li>“Our project team includes members from the <a href="">Jupyter</a> and <a href="">Pangeo</a> communities, with representation across the geosciences including climate modeling, water resource applications, and geophysics. Three active research projects, one in each domain, will motivate developments in the Jupyter and Pangeo ecosystems. Each of these research applications demonstrates aspects of a research workflow which requires scalable, interactive computational tools.”</li> <li>“The adoption of open languages such as Python and the coalescence of communities of practice around open-source tools, is visible in nearly every domain of science. This is a fundamental shift in how science is conducted and shared.”</li> <li>Geoscience use cases <ul> <li>climate data analysis</li> <li>hydrologic modeling</li> <li>geophysical inversions</li> </ul></li> <li>User-Centered Development <ul> <li>data discovery</li> <li>scientific discovery through interactive computing</li> <li>established tools and data visualization</li> <li>using and managing shared computational infrastructure</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Asynchronous Django</strong></a></p> <ul> <li>via Jose Nario</li> <li>Python compatibility <ul> <li>Django 3.0 supports Python 3.6, 3.7, and 3.8. We highly recommend and only officially support the latest release of each series</li> <li>The Django 2.2.x series is the last to support Python 3.5.</li> </ul></li> <li>Other items but</li> <li>Big news: <strong>ASGI support</strong></li> <li>Django 3.0 begins our journey to making Django fully async-capable by providing support for running as an ASGI application.</li> <li>This is in addition to our existing WSGI support. Django intends to support both for the foreseeable future.</li> <li>Note that as a side-effect of this change, Django is now aware of asynchronous event loops and will block you calling code marked as “async unsafe” - such as ORM operations - from an asynchronous context.</li> </ul> <p><strong>Brian #5</strong>: <a href=""><strong>The 1x Engineer</strong></a></p> <ul> <li><em>Fun take on 10x. List actually looks like probably a 3-4x to me, maybe even 8x or more. How high does this scale go anyway?</em></li> <li>non-exhaustive list qualities, here’s a few. <ul> <li>Has a life outside engineering.</li> <li>Writes code that others can read.</li> <li>Doesn't act surprised when someone doesn’t know something.</li> <li>Asks for help when they need it.</li> <li>Is able to say "I don't know."</li> <li>Asks questions.</li> <li>Constructively participates in code reviews.</li> <li>Can collaborate with others.</li> <li>Supports code, even if they did not write it.</li> <li>Can feel like an impostor at times.</li> <li>Shares knowledge.</li> <li>Never stops learning. <ul> <li><em>[obviously listens to Python Bytes, Talk Python, and Test &amp; Code]</em></li> </ul></li> <li>Is willing to leave their comfort zone.</li> <li>Contributes to the community.</li> <li>Has productive and unproductive days.</li> <li>Doesn't take themselves too seriously.</li> <li>Fails from time to time.</li> <li>Has a favorite editor, browser, and operating system, but realizes others do too.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Sunsetting Python 2</strong></a></p> <ul> <li>January 1, 2020, will be the day that we sunset Python 2</li> <li>Why are you doing this? We need to sunset Python 2 so we can help Python users.</li> <li><strong>How long is it till the sunset date?</strong> <a href=""></a> will tell you.</li> <li><strong>What will happen if I do not upgrade by January 1st, 2020?</strong> If people find catastrophic security problems in Python 2, or in software written in Python 2, then most volunteers will not help fix them.</li> <li><strong>I wrote code in Python 2. How should I port it to Python 3?</strong> Please read <a href="">the official "Porting Python 2 Code to Python 3" guide</a>.</li> <li><strong>I didn't hear anything about this till just now. Where did you announce it?</strong> We talked about it at software conferences, on the Python announcement mailing list, on official Python blogs, in textbooks and technical articles, on social media, and to companies that sell Python support.</li> </ul> <p><strong>Extras</strong></p> <p>Brian:</p> <ul> <li>working on a Portland Westside Python Meetup, info will be at <a href=""></a> <ul> <li>Hoping to get something ready for Oct. But… if not, hopefully by Nov.</li> </ul></li> </ul> <p>Michael:</p> <ul> <li><a href=""><strong>Humble Level Up Your Python Bundle</strong></a></li> </ul>
Sep 18, 2019
#147 Mocking out AWS APIs
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>rapidtables</strong></a></p> <ul> <li>“<strong>rapidtables …</strong> converts lists of dictionaries to pre-formatted tables. And it does the job as fast as possible.”</li> <li>Also can do color formatting if used in conjunction with <code>termcolor.colored</code>, but I’m mostly excited about really easily generating tabular data with <code>print</code>.</li> <li>Can also format to markdown or reStructured text, and can do alignment, … </li> </ul> <p><strong>Michael #2</strong>: <a href=""><strong>httpx</strong></a></p> <ul> <li>A next generation HTTP client for Python. 🦋</li> <li>HTTPX builds on the well-established usability of <code>requests</code>, and gives you: <ul> <li>A requests-compatible API.</li> <li>HTTP/2 and HTTP/1.1 support.</li> <li>Support for <a href="">issuing HTTP requests in parallel</a>. <em>(Coming soon)</em></li> <li>Standard synchronous interface, but <a href="">with</a> <code>[async](</code><a href="">/</a><code>[await](</code> <a href="">support if you need it</a>.</li> <li>Ability to <a href="">make requests directly to WSGI or ASGI applications</a>. <ul> <li>This is particularly useful for two main use-cases: <ul> <li>Using <code>httpx</code> as a client, inside test cases.</li> <li>Mocking out external services, during tests or in dev/staging environments.</li> </ul></li> </ul></li> <li>Strict timeouts everywhere.</li> <li>Fully type annotated.</li> <li>100% test coverage.</li> </ul></li> <li>Lovely support for “<a href="">parallel requests</a>” without full <code>asyncio</code> (at the API level). <ul> <li>Also pairs with async / await with async client.</li> </ul></li> <li>Plus all the requests features</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Quick and dirty mock service with Starlette</strong></a></p> <ul> <li>Matt Layman</li> <li>Mock out / fake a third party service in a testing environment.</li> <li>Starlette looks fun, but the process can be used with other API producing server packages.</li> <li>We tell people to do things like this all the time, but there are few examples showing how to.</li> <li>This example also introduces a delay because the service used in production takes over a minute and part of the testing is to make sure the system under test handles that delay gracefully.</li> <li>Very cool, easy to follow write up. (Should probably have Matt on a <a href="">Test &amp; Code</a> episode to talk about this strategy.)</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Mocking out AWS APIs</strong></a></p> <ul> <li>via Giuseppe Cunsolo</li> <li>A library that allows you to easily mock out tests based on AWS infrastructure.</li> <li>Lovely use of a decorator to mock out S3</li> <li>Moto isn't just for Python code and it isn't just for S3. Look at the <a href="">standalone server mode</a> for more information about running Moto with other languages.</li> <li>Be sure to check out <a href="">very important note</a>.</li> </ul> <p><strong>Brian #5</strong>: <a href=""><strong>μMongo: sync/async ODM</strong></a></p> <ul> <li>“μMongo is a Python MongoDB ODM. It inception comes from two needs: the lack of async ODM and the difficulty to do document (un)serialization with existing ODMs.”</li> <li>works with common mongo drivers such as PyMongo, TxMongo, motor_asyncio, and mongomock. (Hadn’t heard of <a href="">mongomock</a> before, I’ll have to try that out.)</li> <li>Note: We’ve discussed <a href="">MongoEngine</a> before. (I’m curious what Michael has to say about uMongo.)</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Single Responsibility Principle</strong></a> <a href=""><strong>in Python</strong></a></p> <ul> <li>via Tyler Matteson</li> <li>I’m a big fan of the <a href="">SOLID principles</a></li> <li>They even come in demotivator style posters <ul> <li><a href=";;docid=No884kr6Vq4XOM&amp;tbnid=fO846X2JegtChM%3A&amp;vet=10ahUKEwjx_rWazLXkAhWEop4KHYLSADIQMwhDKAAwAA..i&amp;w=484&amp;h=484&amp;client=firefox-b-1-d&amp;bih=901&amp;biw=1677&amp;q=SOLID%20principles%20calendar&amp;ved=0ahUKEwjx_rWazLXkAhWEop4KHYLSADIQMwhDKAAwAA&amp;iact=mrc&amp;uact=8">DI</a></li> <li><a href=";;docid=No884kr6Vq4XOM&amp;tbnid=US_KS_hI72yNQM%3A&amp;vet=10ahUKEwjx_rWazLXkAhWEop4KHYLSADIQMwhEKAEwAQ..i&amp;w=485&amp;h=484&amp;client=firefox-b-1-d&amp;bih=901&amp;biw=1677&amp;q=SOLID%20principles%20calendar&amp;ved=0ahUKEwjx_rWazLXkAhWEop4KHYLSADIQMwhEKAEwAQ&amp;iact=mrc&amp;uact=8">Liskov Substitution Principle</a></li> <li><a href=";;docid=k7BTk5K9fRkuhM&amp;tbnid=zBs-qf2LFzsZAM%3A&amp;vet=10ahUKEwjx_rWazLXkAhWEop4KHYLSADIQMwhFKAIwAg..i&amp;w=485&amp;h=484&amp;client=firefox-b-1-d&amp;bih=901&amp;biw=1677&amp;q=SOLID%20principles%20calendar&amp;ved=0ahUKEwjx_rWazLXkAhWEop4KHYLSADIQMwhFKAIwAg&amp;iact=mrc&amp;uact=8">Open/Closed Principle</a></li> <li><a href=";;docid=zOwSzU4-EL6BoM&amp;tbnid=iIl4YkbBFDejHM%3A&amp;vet=10ahUKEwjx_rWazLXkAhWEop4KHYLSADIQMwhGKAMwAw..i&amp;w=728&amp;h=546&amp;client=firefox-b-1-d&amp;bih=901&amp;biw=1677&amp;q=SOLID%20principles%20calendar&amp;ved=0ahUKEwjx_rWazLXkAhWEop4KHYLSADIQMwhGKAMwAw&amp;iact=mrc&amp;uact=8"><strong>Single Responsibility Principle</strong></a></li> <li><a href=";;docid=zOwSzU4-EL6BoM&amp;tbnid=NvBTSxYKM_5BTM%3A&amp;vet=10ahUKEwjx_rWazLXkAhWEop4KHYLSADIQMwhHKAQwBA..i&amp;w=728&amp;h=546&amp;client=firefox-b-1-d&amp;bih=901&amp;biw=1677&amp;q=SOLID%20principles%20calendar&amp;ved=0ahUKEwjx_rWazLXkAhWEop4KHYLSADIQMwhHKAQwBA&amp;iact=mrc&amp;uact=8">Interface Segregation Principle</a></li> </ul></li> <li>This article will guide you through a <strong>complex process of writing simple code</strong>.</li> </ul> <p><strong>Extras</strong></p> <p>Michael:</p> <ul> <li><a href="">Code Challenge Bite 220. Analyzing @pythonbytes RSS feed</a></li> </ul> <p><strong>Jokes</strong> </p> <ul> <li>Q: What do you get when you cross a computer and a life guard?</li> <li><p>A: A screensaver!</p></li> <li><p>Q: What do you get when you cross a computer with an elephant?</p></li> <li>A: Lots of memory!</li> </ul> <p>via <a href=""></a> </p> <p>Anti-joke (we ready for those yet?): A Python developer, a PHP developer, a C# developer, and a Go developer went to lunch together. They had a nice lunch and got along fine.</p>
Sep 11, 2019
#146 Slay the dragon, learn the Python
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>Special guest: Trey Hunner</p> <p><strong>Brian #1:</strong> <a href=""><strong>Positional-only arguments in Python</strong></a></p> <ul> <li>by Sanket</li> <li>Feature in 3.8</li> <li>“To specify arguments as positional-only, a <code>/</code> marker should be added <em>after</em> all those arguments in the function definition. “</li> <li>Great example of a <code>pow(x, y, /, mod=None)</code> function where the names <code>x</code> and <code>y</code> are meaningless and if given in the wrong order would be confusing. </li> </ul> <p><strong>Trey #2:</strong> <a href=""><strong>django-stubs</strong></a></p> <ul> <li>Type checking for Django!</li> <li>It’s new and from my very quick testing on my Django site it definitely doesn’t work with everything yet, but it’s promising</li> <li>I don’t use type annotations in Django yet, but I very well might eventually</li> </ul> <p><strong>Michael #3:</strong> <a href=""><strong>CodeCombat</strong></a></p> <ul> <li>Super fun game for learning to code</li> <li><em>Real</em> code but incredibly easy coding</li> <li>Can subscribe or use the free tier</li> <li>Just got <a href="">$6M VC funding</a></li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>Four Use Cases for When to Use Celery in a Flask Application</strong></a></p> <ul> <li>or really any web framework</li> <li>by Nick Janetakis</li> <li>“Celery helps you run code asynchronously or on a periodic schedule which are very common things you'd want to do in most web projects.”</li> <li>examples: <ul> <li>sending emails out</li> <li>connecting to 3rd party APIs. </li> <li>performing long running tasks. Like, really long. </li> <li>Running tasks on a schedule. </li> </ul></li> </ul> <p><strong>Trey #5:</strong> <a href=""><strong>pytest-steps</strong></a></p> <ul> <li>Created by smarie</li> <li>Can use a generator syntax with yield statements to break a big test up into multiple named “steps” that’ll show up in your pytest output</li> <li>If one step fails, the rest of the steps will be skipped by default</li> <li>You can also customize it to make optional steps, which aren’t required for future steps to run, or steps which depend on other optional steps explicitly</li> <li>The documentation shows a lot of different ways to use it, but the generator approach looks by far the most readable to me (also state is shared between steps with this approach whereas the others require some fancy state-capturing object which looks confusing to me)</li> <li>I haven’t tried this, but my use case would be my end-to-end/functional tests, which would work great with steps because I’m often using Selenium to navigate between a number of different pages and forms, one click at a time.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>docassemble</strong></a></p> <ul> <li>Created by Jonathan Pyle</li> <li>A free, open-source expert system for guided interviews and document assembly, based on Python, YAML, and Markdown.</li> <li>Features <ol> <li>WYSIWYG: Compose your templates in <a href=""></a><a href="">.docx</a> (with help of a <a href=""></a><a href="">Word Add-in</a>) or <a href=""></a><a href="">.pdf</a> files.</li> <li>Signatures: Gather touchscreen signatures and embed them in documents.</li> <li>Live chat: Assist users in real time with live chat, screen sharing, and remote screen control.</li> <li>AI: Use machine learning to process user input.</li> <li>SMS: Send text messages to your users</li> <li>E-mail: Send and receive e-mails in your interviews.</li> <li>OCR: Use optical character recognition to process images uploaded by the user.</li> <li>Multilingual: Offer interviews in multiple languages.</li> <li>Multiuser: Develop applications that involve more than one user, such as mediation or counseling interviews.</li> <li>Extensible: Use the power of Python to extend the capabilities of your interviews.</li> <li>Open: Package your interviews and use GitHub and PyPI to share your work with the docassemble user community.</li> <li>Background Tasks: Do things behind the scenes of the interview, even when the user is not logged in.</li> <li>Scalable: Deploy your interviews on multiple machines to handle high traffic.</li> <li>Secure: Protect user information with server-side encryption, two-factor authentication, document redaction, and other security features.</li> <li>APIs: Integrate with third-party applications using the API, or send your interviews input and extract output using code.</li> </ol></li> </ul> <p><strong>Extras</strong></p> <p>Michael:</p> <ul> <li><a href="">PyPI closes in on 200k</a></li> <li><a href="">NumPy 1.17.0 released</a></li> <li><a href="">Python 3.8.0b4 is out</a></li> </ul> <p><strong>Joke</strong></p> <p>via Avram Lubkin</p> <p>Knock! Knock! Who's there? Recursive function. Recursive function who? Knock! Knock!</p> <p>Nice. to get that joke, you’ll have to understand recursion. to understand recursion, </p> <ul> <li>either google “recursion”, and click on “did you mean “recursion””</li> <li>learn it in small steps. step one, recursion</li> </ul> <p>text conversation:</p> <ul> <li>first person: “Hey, what’s your address?”</li> <li>second: gives IP address</li> <li>first: “No. Your local address”</li> <li>second:</li> <li>first: “No. Your physical address”</li> <li>second: gives MAC address</li> </ul>
Sep 08, 2019
#145 The Python 3 “Y2K” problem
<p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <p><strong>Special guests</strong></p> <ul> <li><a href=""><strong>Matt Harrison</strong></a></li> <li><a href=""><strong>Anthony Sottile</strong></a></li> </ul> <p>Michael #1: <a href=""><strong>friendly-traceback</strong></a></p> <ul> <li>via Jose Carlos Garcia (I think 🙂 )</li> <li>Aimed at Python beginners: replacing standard traceback by something easier to understand</li> <li>Shows help for exception type</li> <li>Shows local variable values</li> <li>Shows code in a cleaner form with more context</li> <li><a href="">3 ways to install</a> <ul> <li>As an exception hook</li> <li>Explicit explain</li> <li>When running an app</li> </ul></li> </ul> <p><strong>Matt #2: <a href="">Pandas Users Survey</a></strong> </p> <ul> <li>Most use it almost everyday but have less than 2 years experience</li> <li>Linux 61%, Windows 60%, Mac 42%</li> <li>93% Python 3</li> </ul> <p><strong>Anthony #3: python3 “Y2K” problem (python3.10 / python4.0)</strong></p> <ul> <li>with python3.8 close to release and python3.9 right around the corner, what comes after?</li> <li>both python3.10 and python4.0 present some problems <ul> <li><code>sys.version[:3]</code> which will suddenly report <code>'``3.1``'</code> in 3.10</li> <li>a lot of code (including <code>six.PY3</code>!) uses <code>sys.version_info[0] == 3</code> which will suddenly be false in python4.0 (and start running python2 code!)</li> </ul></li> <li>early-to-mid 2020 we should start seeing the next version in the wild as <a href="">python3.9 reaches beta</a></li> <li>easy ways to start testing this early: <ul> <li><a href="">python3.10</a> - a build of cpython for ubuntu with the version number changed</li> <li><a href="">flake8-2020</a> - a flake8 plugin which checks for these common issues-</li> </ul></li> </ul> <p>Michael #4: <a href=""><strong>pypi research</strong></a></p> <ul> <li>via Adam (Codependent Codr)</li> <li>Really interesting research paper on the current state of Pypi from a couple authors at the University of Michigan: "An Empirical Analysis of the Python Package Index" -</li> <li>Comprehensive empirical summary of the Python Package Repository, PyPI, including both package metadata and source code covering 178,592 packages, 1,745,744 releases, 76,997 contributors, and 156,816,750 import statements. </li> <li>We provide counts and trends for packages, releases, dependencies, category classifications, licenses, and package imports, as well as authors, maintainers, and organizations.</li> <li>Within PyPI, we find that the growth of the repository has been robust under all measures, with a compound annual growth rate of 47% for active packages, 39% for new authors, and 61% for new import statements over the last 15 years. </li> <li>In 2005, there were 96 active packages, 96!</li> <li>MIT is the most common license</li> <li>(Matt) I saw this and was surprised at most commonly used libraries. What do you think the most common 3rd party library is?</li> </ul> <p><strong>Matt #5: <a href="">DaPy</a></strong></p> <ul> <li>“Pandas for humans” - Matt’s words</li> <li>Has portions of pandas, scikit-learn, yellowbrick, and numpy</li> <li>Designed for “data analysis, not for coders”</li> </ul> <p>Anthony #6: <a href=""><strong>python-remote-pdb</strong></a></p> <ul> <li>very small over-the-network remote debugger</li> <li>thin wrapper around <code>pdb</code> in a single file (easy to drop the file on <code>PYTHONPATH</code> if you can’t <code>pip install</code>)</li> <li>not as fully featured as other remote debuggers such as <code>pudb</code> / <code>rpdb</code> / pycharm’s debugger but very easy to drop in</li> <li>fully supports <code>[breakpoint()](</code> (python3.7+ or via <a href="">future-breakpoint</a>)</li> <li>access <code>pdb</code> via <code>telnet</code> / <code>nc</code> / <code>socat</code></li> <li>I’m using it to debug a text editor I’m writing to learn <code>curses</code>!</li> </ul> <p>Extras:</p> <p>Michael:</p> <ul> <li><a href="">Hacker Gets $12,000 In Parking Tickets After 'NULL' License Plate Trick Backfires</a></li> <li><a href="">PyCon 2020 site is up</a></li> </ul> <p>Matt</p> <ul> <li><a href=""></a> - My new course on Machine Learning with XGBoost</li> </ul> <p>Anthony:</p> <ul> <li><a href=""></a> “like selenium webdriver for the terminal”</li> </ul> <p>Jokes:</p> <p>Michael: Two mathematicians are sitting at a table in a pub having an argument about the level of math education among the general public.</p> <p>The one defending overall math knowledge gets up to go to the washroom. On the way back, he encounters their waitress and says, "I'll add an extra $10 to your tip, if you'll answer a question for me when I ask it. All you have to say is 'x-squared'." She agrees.</p> <p>A few minutes later the populist mathematician says to his buddy, "I'll bet you $20 that even our waitress can tell us the integral of 2x." The cynic agrees to the bet.</p> <p>So the schemer beckons the waitress to their table and asks the question, to which she replies "x-squared". As he begins to gloat and demand his winnings, the waitress continues, "Plus a constant."</p> <p>Anthony: I had a golang joke prepared, but then I <code>panic()</code>d</p>
Aug 31, 2019
#144 Are you mocking me? It won't work!
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Chris #1:</strong> <a href=""><strong>Why your mock doesn’t work</strong></a></p> <ul> <li>Ned Batchelder</li> <li>TDD is an important practice for development, and as my team is finding out, mocking objects is not as easy at it seems at first.</li> <li>I love that Ned gives an overview of how Mock works</li> <li>But also gives two resources to show you alternatives to Mock, when you really don’t need it.</li> <li>From reading these articles and video, I’ve learned that it’s hard to make mocks but it’s important to: <ul> <li>Create only one mock for each object you’re mocking</li> <li>that mocks only what you need</li> <li>have tests that run the mock against your code and your mock against the third party </li> </ul></li> </ul> <p><strong>Mahmoud #2</strong>: <a href=""><strong>Vermin</strong></a></p> <ul> <li>By Morten Kristensen</li> <li>Rules-based Python version compatibility detector</li> <li>caniuse is cool, but it’s based on classifiers. When it comes to your own code, it’ll only tell you what you tell it.</li> <li>If you’ve got legacy libraries, or like most companies, an application, then you’ll need something more powerful.</li> <li>Vermin tells you the minimum compatible Python version, all the way down to the module and even function level.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>The nonlocal statement in Python</strong></a></p> <ul> <li>Abhilash Raj</li> <li>When <code>global</code> is too big of a hammer.</li> <li>This doesn’t work:</li> </ul> <pre><code> def function(): x = 100 def incr(y): x = x + y incr(100) </code></pre> <ul> <li>This does:</li> </ul> <pre><code> def function(): x = 100 def incr(y): nonlocal x x = x + y incr(100) print(x) </code></pre> <p><strong>Chris #4:</strong> <strong><a href=""></a></strong></p> <ul> <li>Brett Cannon</li> <li>Microsoft Azure improves python support <ul> <li>2 key points about the new Python support in Azure Functions: <ul> <li>it's debuting w/ 3.6, but 3.7 support is actively being worked on and 3.8 support won't take nearly as long, and </li> <li>native async/await support!</li> </ul></li> </ul></li> </ul> <p><strong>Mahmoud #5</strong>: <a href=""><strong>Awesome Python Applications</strong></a> update</p> <ul> <li>Presented at PyBay 2019</li> <li>Slides/summary (video forthcoming): <a href=""></a></li> <li>250+ applications, dating back to 1998 (mailman, gedit)</li> <li>95% of applications have commits in 2019</li> <li>65% of applications support Python 3 (even the ones with a long history!)</li> <li>Other interesting findings</li> <li>Presenting these findings and more at PyGotham 2019. NYC in early October.</li> </ul> <p><strong>Brian #6:</strong> <a href=""><strong>pre-commit now has a quick start guide</strong></a></p> <ul> <li>Wanna use pre-commit but don’t know how to start? Here ya go!</li> <li>Runs through <ul> <li>install</li> <li>configuration</li> <li>installing hooks</li> <li>running hooks against your project</li> </ul></li> <li>I’d like to add <ul> <li>Add hooks to your project one at a time</li> <li>For each new hook <ul> <li>add to <code>pre-commit-config.yml</code></li> <li>run <code>pre-commit install</code> to install hook</li> <li>run <code>pre-commit run</code> <code>--``all-files</code></li> <li>review changes made to your project <ul> <li>if good, commit</li> <li>if bad</li> <li>revert</li> <li>modify config of tools, such as <code>pyproject.toml</code> for black, <code>.flake8</code> for flake8, etc.</li> <li>try again</li> </ul></li> </ul></li> </ul></li> </ul> <p><strong>Extras</strong></p> <p>Chris: </p> <ul> <li><a href="">Humble Bundle by No Starch supports the Python Software Foundation</a></li> <li><a href=""></a> released Newbie Bites… challenges that are intended for people brand new to python. [[direct link](]</li> </ul> <p>Mahmoud:</p> <ul> <li>PyGotham 2019 October (Maintainers Conf in Washington DC, too)</li> <li>Real Python Pandas course</li> </ul> <p>Brian:</p> <ul> <li><a href=""></a> shows 360 of the top downloaded Python packages are all Python 3 ready.</li> </ul> <p><strong>Jokes</strong> </p> <ul> <li>I was looking for some programming one liners online; looked on a reddit thread; read a great answer; which was “any joke can be a one-liner with enough semicolons.”</li> <li>A SQL statement walks into to a bar and up to two tables and asks, “Mind if I join you?”</li> </ul>
Aug 23, 2019
#143 Spike the robot, powered by Python!
<p>Special guest: <a href=""><strong>Kelly Schuster-Paredes</strong></a></p> <p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Keynote: Python 2020 - Łukasz Langa - PyLondinium19</strong></a></p> <ul> <li>Enabling Python on new platforms is important.</li> <li>Python needs to expand further than just CPython. <ul> <li>Web, 3D games, system orchestration, mobile, all have other languages that are more used. Perhaps it’s because the full Python language, like CPython in full is more than is needed, and a limited language is necessary.</li> </ul></li> <li>MicroPython and CircuitPython are successful. <ul> <li>They are limited implementations of Python</li> </ul></li> <li>Łukasz talks about many parts of Python that could probably be trimmed to make targeted platforms very usable without losing too much. </li> <li>It’d be great if more projects tried to implement Python versions for other platforms, even if the Python implementation is limited.</li> </ul> <p><strong>Kelly #2</strong>: <strong><a href="">Mu Editor</a></strong></p> <ul> <li>by <a href="">Nicholas Tollervey</a></li> <li>Lots of updates happening to the Code with Mu software</li> <li>Mu is a Python code editor for beginner programmers <ul> <li>originally created as a contribution from the <a href="">Python Software Foundation</a> for the BBC’s <a href="">micro:bit project</a></li> </ul></li> <li>Code with Mu presented at EuroPython and shared a lot of interesting updates and things in the alpha version of Mu, <a href="">available on code with Mu</a> website.</li> <li>Mu is a modal editor: <ul> <li>BBC Microbit</li> <li>Circuit Python</li> <li>ESP Micropython</li> <li>Pygame Zero</li> <li>Python 3 <ul> <li>Tiago Monte’s recorded presentation at EuroPython</li> <li>Game with Turtle</li> </ul></li> <li>Flask — <a href="">release notes</a></li> </ul></li> <li><a href="">Made with Mu at EuroPython</a> videos</li> <li>Hot off the press: Nick just released <a href="">Pypercard</a> a <a href="">HyperCard inspired</a> GUI framework for BEGINNER developers in Python based off of <a href="">Adafruit’s release</a>. <ul> <li>It is a “PyperCard is a <a href="">HyperCard</a> inspired <a href="">Pythonic</a> and deliberately constrained <a href="">GUI</a> framework for beginner programmers.</li> <li>linked repos on <a href="">GitHub</a>.</li> <li>module re-uses the JSON specification used to create HyperCard</li> <li>The concept allows user to “create Hypercard like stacks of states” to allow beginner coders to create choose their own adventure games. </li> </ul></li> </ul> <p><strong>Michael #3</strong>: <a href=""><strong>Understanding the Python Traceback</strong></a></p> <ul> <li>by <a href="">Chad Hansen</a></li> <li>The Python traceback has a wealth of information that can help you diagnose and fix the reason for the exception being raised in your code.</li> <li>What do we learn right away? <ul> <li>The type of error</li> <li>A description of the error (hopefully, sometimes)</li> <li>The line of code the error occurred on</li> <li>The call stack (filenames, line numbers, and module names)</li> <li>If the error happened while handling another error</li> </ul></li> <li>Read from bottom to top — that was weird to me</li> <li>Most common error? AttributeError: 'NoneType' object has no attribute 'an_attribute'</li> <li>Article talks about other common errors</li> <li>Are you creating custom exceptions to make your packages more useful?</li> </ul> <p><strong>Brian #4:</strong> <strong>My oh my,</strong> <a href=""><strong>flake8-mypy</strong></a> <strong>and</strong> <a href=""><strong>pytest-mypy</strong></a></p> <ul> <li>contributed by Ray Cote via email</li> <li>“For some reason, I continually have problems running mypy, getting it to look at the correct paths, etc. However, when I run it from flake8-mypy, I'm getting reasonable, actionable output that is helping me slowly type hint my code (and shake out a few bugs in the process). There's also a pytest-mypy, which I've not yet tried. “ - Ray</li> <li><a href=""><strong>flake8-mypy</strong></a> <em>*</em>* <ul> <li>Maintained by Łukasz Langa</li> <li>“The idea is to enable limited type checking as a linter inside editors and other tools that already support <em>Flake8</em> warning syntax and config.”</li> </ul></li> <li><a href=""><strong>pytest-mypy</strong></a> <ul> <li>Maintained by Dan Bader and David Tucker</li> <li>“Runs the mypy static type checker on your source files as part of your pytest test runs.” <ul> <li>Remind me to do a PR against the README to make pytest lowercase. </li> </ul></li> </ul></li> </ul> <p><strong>Kelly #5</strong>: <strong><a href="">Lego Education and Spike</a></strong></p> <ul> <li>In March of this year, Lego Education gave news of a new robot being released since the EV3 released of Mindstorms in 2013. <ul> <li>Currently the EV3 Mindstorm can be coded with Python and it is assumed that Spike Prime can be as well. </li> </ul></li> <li>The current EV3 robots can currently be coded in python thanks to <a href="">Nigel Ward. He created a site back in 2016</a> or earlier; through a program called the EV3Dev project. <ul> <li><a href="">ev3dev</a> is a <a href="">Debian Linux</a>-based operating system </li> </ul></li> <li>Until recently, Lego had not endorsed the use of Python or had they released documentation. <ul> <li>Lego released a Getting started with EV3 MicroPython <a href="">59 page guide Version 1.0.0</a></li> <li>EV3 MicroPython runs on top of ev3dev with a new Pybricks MicroPython runtime and library.</li> <li>has its own Visual Studio Code extension</li> <li>no need for terminal</li> <li>Has instruction and lists of different features and classes used to program the PyBricks API- A python wrapper for the Databricks Rest API. <ul> <li>Pybricks is on <a href="">GitHub</a> from one contributor, <a href=""><strong>Sebastien Thomas</strong></a> <strong>under MIT license</strong></li> <li><a href="">David Lechner</a>, <a href="">Laurens Valk</a>, and <a href="">Anton Vanhoucke</a> are contributors of the Lego MicroPython release.</li> </ul></li> </ul></li> <li>This opens up opportunities for students that compete in the First Lego League Competition to code in Python.</li> <li><a href="">Example code for the Gyrobot</a></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Python 3 at Mozilla</strong></a></p> <ul> <li>From January 2019.</li> <li>Mozilla uses a lot of Python.</li> <li>In mozilla-central there are over 3500 Python files (excluding third party files), comprising roughly 230k lines of code. </li> <li>Additionally there are 462 repositories labelled with Python in the Mozilla org on Github</li> <li>That’s a lot of Python, and most of it is <strong>Python 2</strong>.</li> <li>But before tackling those questions, I want to address another one that often comes up right off the bat: Do we need to be 100% migrated by Python 2’s EOL?</li> <li>No. But punting the migration into the indefinite future would be a big mistake: <ul> <li>Python 2 will no longer receive security fixes.</li> <li>All of the third party packages we rely on (and there are a lot of them) will also stop being supported</li> <li>Delaying means more code to migrate</li> <li>Opportunity cost: Python 3 was first released in 2008 and in that time there have been a huge number of features and improvements that are not available in Python 2.</li> </ul></li> <li>The best time to get serious about migrating to Python 3 was five years ago. The second best time is now.</li> <li>Moving to Python 3</li> <li>We stood up some linters. <ul> <li>One linter that makes sure Python files can at least get imported in Python 3 without failing</li> <li>One that makes sure Python 2 files use appropriate <code>__future__</code> statements to make migrating that file slightly easier in the future.</li> </ul></li> <li>Pipenv &amp; poetry &amp; <a href="">Jetty</a>: a little experiment I’ve been building. It is a very thin wrapper around Poetry</li> </ul> <p><strong>Extras</strong></p> <p>Brian:</p> <ul> <li><a href="">Python 3.8.0b3</a> <ul> <li>“We strongly encourage maintainers of third-party Python projects to test with 3.8 during the beta phase and report issues …”</li> </ul></li> </ul> <p>Michael:</p> <ul> <li>pipx now has <a href=""><strong>shell completions</strong></a></li> </ul> <p>Kelly:</p> <ul> <li><a href="">Teaching Python</a> podcast</li> </ul> <p><strong>Jokes</strong> </p> <ul> <li>via Real Python and Nick Spirit</li> <li>Python private method → Joke <a href=""><strong>cartoon image</strong></a>.</li> </ul>
Aug 14, 2019
#142 There's a bandit in the Python space
<p>Special guest: <strong><a href="">Brett Thomas</a></strong></p> <p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Writing sustainable Python scripts</strong></a></p> <ul> <li>Vincent Bernat</li> <li>Turning a quick Python script into a maintainable bit of software.</li> <li>Topics covered: <ul> <li>Documentation as a docstring helps future users/maintainers know what problem you are solving.</li> <li>CLI arguments with defaults instead of hardcoded values help extend the usability of the script.</li> <li>Logging. Including debug logging (and how to turn them on with CLI arguments), and system logging for unattended scripts.</li> <li>Tests. Simple doctests, and pytest tests utilizing parametrize to have one test and many test cases.</li> </ul></li> </ul> <p><strong>Brett #2:</strong> <strong>Static Analysis and</strong> <a href=""><strong>Bandit</strong></a></p> <p><strong>Michael #3:</strong> <a href=""><strong>jupyter-black</strong></a></p> <ul> <li>Black formatter for Jupyter Notebook </li> <li>One of the big gripes I have about these online editors is their formatting (often entirely absent)</li> <li>Then the extension provides <ul> <li>a toolbar button</li> <li>a keyboard shortcut for reformatting the current code-cell (default: Ctrl-B)</li> <li>a keyboard shortcut for reformatting whole code-cells (default: Ctrl-Shift-B)</li> </ul></li> </ul> <p><strong>Brian #4:</strong> <strong>Report Generation workflow with papermill, jupyter, rclone, nbconvert, …</strong></p> <ul> <li>Chris Moffitt articles</li> <li><a href="">Automated Report Generation with Papermill: Part 1</a></li> <li><a href="">Automated Report Generation with Papermill: Part 2</a></li> <li>Jupyter Notebooks used to create a report with pandas and matplotlib</li> <li>nbconvert to create an html report</li> <li>Papermill to parametrize the process with different data, and execute the notebook</li> <li>Copy the reports to shared cloud folders using Rclone.</li> <li>Set up a process to automate everything.</li> <li>Hook it up to cron to run regularly</li> </ul> <p><strong>Brett #5</strong>: Rant on time deltas</p> <ul> <li><code>datetime.timedelta(months=1)</code> # Boom, too bad.</li> <li>Use: <strong><a href=""></a></strong></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>How — and why — you should use Python Generators</strong></a></p> <ul> <li>by Radu Raicea</li> <li>Generator functions allow you to declare a function that behaves like an iterator. </li> <li>They allow programmers to make an iterator in a fast, easy, and clean way.</li> <li>They only compute it when you ask for it. This is known as <a href="">lazy evaluation</a>.</li> <li>If you’re not using generators, you’re missing a powerful feature</li> <li>Often they result in simpler code than with lists and standard functions</li> </ul> <p><strong>Extras</strong></p> <p>Brian:</p> <ul> <li><a href=""><strong>PyPI now supports uploading via API token</strong></a> <ul> <li>also on <a href="">Test PyPI</a></li> </ul></li> </ul> <p>Michael:</p> <ul> <li><a href=""><strong>Chocolatey package manager</strong></a> on windows via <a href="">Prayson Daniel</a></li> <li>GvM’s Next <a href=""><strong>PEG article</strong></a></li> </ul> <p><strong>Jokes</strong> </p> <p>A good programmer is someone who always looks both ways before crossing a one-way street.</p> <p>(reminds me of another joke: Adulthood is like looking both ways before crossing the street, then getting hit by an airplane)</p> <p><a href=""><strong>Little bobby tables</strong></a></p>
Aug 06, 2019
#141 Debugging with f-strings coming in Python 3.8
<p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Debugging with f-strings in Python 3.8</strong></a></p> <ul> <li>We’ve talked about the walrus operator, <code>:=</code>, but not yet “debug support for f-strings”</li> <li>this: <code>print(f'foo={foo} bar={bar}')</code></li> <li>can change to this: <code>print(f'{foo=} {bar=}')</code></li> <li>and if you don’t want to print with <code>repr()</code> you can have <code>str()</code> be used with <code>!s</code>. <ul> <li><code>print(f'{foo=!s} {bar=!s}')</code></li> </ul></li> <li>also <code>!f</code> can be used for float modifiers:</li> </ul> <pre><code> &gt;&gt;&gt; import math &gt;&gt;&gt; print(f'{math.pi=!f:.2f}') math.pi=3.14 </code></pre> <ul> <li>one more feature, space preservation in the f-string expressions:</li> </ul> <pre><code> &gt;&gt;&gt; a = 37 &gt;&gt;&gt; print(f'{a = }, {a = }') a = 37, a = 37 </code></pre> <p><strong>Michael #2:</strong> <a href=""><strong>Am I "real" software developer yet?</strong></a></p> <ul> <li>by <a href="">Sun-Li Beatteay</a></li> <li>To new programmers joining the field, especially those without CS degrees, it can feel like the title is safe-guarded. Only bestowed on the select that have proven themselves.</li> <li>Sometimes manifests itself as Impostor Syndrome</li> <li>Focused on front-end development as I had heard that HTML, CSS and JavaScript were easy to pick up</li> <li>That was when I decided to <a href="">create a portfolio site for my wife</a>, who was a product designer.</li> <li>Did my best to surround myself with tech culture. <ul> <li>Watched YouTube videos</li> <li>listened to podcasts</li> <li>read blog posts from experienced engineers to keep myself motivated. </li> <li>Daydreamed what it would be like to stand in their shoes.</li> </ul></li> <li>My wife’s website went live in July of that year. I had done it.</li> <li>Could I finally start calling myself something of a <em>Software Engineer</em>? <ul> <li>“Web development isn’t real programming”</li> </ul></li> <li>Spent the next 18 months studying software development full time. I quit my job and moved in with my in-laws — which was a journey in-and-of itself. <ul> <li>Software engineer after 1-2 years? No so fast (says the internet)</li> </ul></li> <li>The solution that I found for myself was simple yet terrifying: talking to people</li> <li>MK: BTW, I don’t really like the term “engineer”</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>De</strong></a><a href=""><strong>bugging with local variables and snoop</strong></a></p> <ul> <li>debugging tools</li> <li>ex: “You want to know which lines are running and which aren't, and what the values of the local variables are.” <ul> <li>Throw a <code>@snoop</code> decorator on a function and the function lines and local variable values will be dumped to stderr during run. Even showing loops a bunch of times.</li> </ul></li> <li>It’s tools to almost debug as if you had a debugger, without a debugger, and without having to add a bunch of logging or print statements.</li> <li>Lots of other use models to allow more focus. <ul> <li>wrap just part of your function with a <code>with snoop</code> block</li> <li>only watch certain local variables.</li> <li>turn off reporting for deep function/block levels.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>New home for Humans</strong></a></p> <ul> <li>This came out of the blue with some trepidation:</li> <li>kennethreitz commented 6 days ago:</li> </ul> <p>In the spirit of transparency, I'd like to (publicly) find a new home for my repositories. I want to be able to still make contributions to them, but no longer be considered the "owner" or "arbiter" or "BDFL" of these repositories.</p> <p>Some notable repos:</p> <ul> <li><a href=""></a></li> <li><a href=""></a></li> <li><a href=""></a></li> <li><a href=""></a></li> <li><a href=""></a></li> <li><p><a href=""></a></p></li> <li><p>Lots of back and forth until <a href=""><strong>Ernest jumped in</strong></a>.</p></li> <li>The Python Software Foundation would like to offer to accept transfers of these repositories into the <a href="">@psf</a> GitHub organization. This organization was recently acquired by the Python Software Foundation and intended to provide administrative backstopping for projects in the ecosystem; existing maintainers of various projects will remain and the PSF staff will be available to manage repositories and teams as necessary.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>The Backwards Commercial License</strong></a></p> <ul> <li>Eran Hammer - open source dev, including hapi.js</li> <li>Interesting idea to make open source projects maintainable</li> <li>Three phases of software lifecycle for some projects: <ul> <li>first: project created to fill a need in one project/team/company, a single use case</li> <li>second: used by many, active community, growing audience</li> <li>three: work feels finished. bug fixes, security issues, minor features continue, but most people can stay on old stable versions</li> </ul></li> <li>During the “done” phase, companies would like to have bug fixes but don’t want to have to keep changing their code to keep up.</li> <li>Idea: commercial license to support old stable versions. <ul> <li>“If you keep up with the latest version, you do not require a license (unless you want the additional benefits it will provide).”</li> <li>“However, very few companies can quickly migrate every time there is a new major release of a core component. Engineering resources are limited and in most cases, are better directed at building great products than upgrading supporting infrastructure. The backwards license provides this exact assurance. You can stay on any version you would like knowing that you are still running supported, well-maintained, and secure code.”</li> <li>“The new commercial license will include additional benefits focused on providing enterprise customers the assurances needed to rely on these critical components for many years to come. “</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Switching Python Parsers?</strong></a></p> <ul> <li>via <a href="">Gi Bi</a>, article by Guido van Rossum</li> <li>Alternative to the home-grown parser generator that I developed 30 years ago when I started working on Python. (That parser generator, dubbed “pgen”, was just about the first piece of code I wrote for Python.)</li> <li>Here are some of the issues with pgen that annoy me. <ul> <li>The “1” in the LL(1) moniker implies that it uses only a single token lookahead, and this limits our ability of writing nice grammar rules.</li> <li>Because of the single-token lookahead, the parser cannot determine whether it is looking at the start of an expression or an assignment.</li> </ul></li> <li>So how does a PEG parser solve these annoyances? By using an infinite lookahead buffer! </li> <li>The typical implementation of a PEG parser uses something called “packrat parsing”, which not only loads the entire program in memory before parsing it, but also allows the parser to backtrack arbitrarily.</li> <li>Why not sooner? Memory! But that is much less of an issue now.</li> <li>My idea now, putting these things together, is to see if we can create a new parser for CPython that uses PEG and packrat parsing to construct the AST directly during parsing, thereby skipping the intermediate parse tree construction, possibly <em>saving</em> memory despite using an infinite lookahead buffer</li> </ul> <p><strong>Extras</strong></p> <p>Brian: </p> <ul> <li>Plone 5.2 <a href=""></a> <ul> <li>Plone is a content management system built on top of Zope, a web application server framework.</li> <li>Plone 5.2 <ul> <li>supports Python 3.6, 3.7, 3.8</li> <li>uses Zope 4, which also support Python 3</li> </ul></li> <li>Multi-year effort <ul> <li><a href="">Interview with Philip Bauer, organizer of 5.2.</a></li> </ul></li> </ul></li> </ul> <p>Michael:</p> <ul> <li><a href="">Building Dab and T-Pose Controlled Lights - Make Art with Python</a> </li> </ul> <p><strong>Jokes</strong> </p> <p>A couple of quick ones:</p> <ul> <li>“What is a whale’s favorite language?” “C” — via Eric Nelson</li> <li>Why does Pythons live on land? Because it is above C-level! — via Jesper Kjær Sørensen @JKSlonester</li> </ul>
Jul 29, 2019
#140 Becoming a 10x Developer (sorta)
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Becoming a 10x Developer : 10 ways to be a better teammate</strong></a></p> <ul> <li>Kate Heddleston</li> <li>“A 10x engineer isn’t someone who is 10x better than those around them, but someone who makes those around them 10x better.” <ol> <li>Create an environment of psychological safety</li> <li>Encourage everyone to participate equally</li> <li>Assign credit accurately and generously</li> <li>Amplify unheard voices in meetings</li> <li>Give constructive, actionable feedback and avoid personal criticism</li> <li>Hold yourself and others accountable</li> <li>Cultivate excellence in an area that is valuable to the team</li> <li>Educate yourself about diversity, inclusivity, and equality in the workplace</li> <li>Maintain a growth mindset</li> <li>Advocate for company policies that increase workplace equality</li> </ol></li> <li>article includes lots of actionable advice on how to put these into practice.</li> <li>examples: <ul> <li>Ask people their opinions in meetings.</li> <li>Notice when someone else might be dominating a conversation and make room for others to speak.</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <strong>quasar &amp;</strong></p> <ul> <li>via Doug Farrell</li> <li><a href=""><strong>Quasar</strong></a> is a Vue.js based framework, which allows you as a web developer to quickly create responsive++ websites/apps in many flavours: <ul> <li>SPAs (Single Page App)</li> <li>SSR (Server-side Rendered App) (+ optional PWA client takeover)</li> <li>PWAs (Progressive Web App)</li> <li>Mobile Apps (Android, iOS, …) through Apache Cordova</li> <li>Multi-platform Desktop Apps (using Electron)</li> </ul></li> <li>Great for python backends</li> <li>tons of vue components</li> <li>But could it be all python? <ul> <li> provides Python bindings for <a href="">Vue.js</a>. It uses <a href="">brython</a> to run Python in the browser.</li> <li>Examples can be found <a href="">here</a>. </li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Regular Expressions 101</strong></a> </p> <ul> <li>We talked about regular expressions in <a href="">episode 138</a></li> <li>Some tools shared with me after I shared a regex joke on twitter, including this one.</li> <li>build expressions for Python and also PHP, JavaScript, and Go</li> <li>put in an example, and build the regex to match</li> <li>explanations included</li> <li>match information including match groups and multiple matches</li> <li>quick reference of all the special characters and what they mean</li> <li>generates code for you to see how to use it in Python</li> <li>Also fun (and shared from twitter): <ul> <li><a href="">Regex Golf</a> <ul> <li>see how far you can get matching strings on the left but not the list on the right. <ul> <li>I got 3 in and got stuck. seems I need to practice some more</li> </ul></li> </ul></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>python-diskcache</strong></a></p> <ul> <li>Caching can be HUGE for perf benefits</li> <li>But memory can be an issue</li> <li>Persistence across executions (e.g. web app redeploy) an issue</li> <li>Servers can be issues themselves</li> <li>Enter the disk! Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.</li> <li>DigitalOcean and many hosts now offer SSD’s be default</li> <li>Unfortunately the file-based cache in Django is essentially broken.</li> <li>DiskCache efficiently makes gigabytes of storage space available for caching. <ul> <li>By leveraging rock-solid database libraries and memory-mapped files, cache performance can match and exceed industry-standard solutions. </li> <li>There's no need for a C compiler or running another process. </li> <li>Performance is a feature </li> <li>Testing has 100% coverage with unit tests and hours of stress.</li> </ul></li> <li>Nice comparison chart</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>The Python Help System</strong></a></p> <ul> <li>Overview of the built in Python help system, <code>help()</code></li> <li>examples to try in a repl <ul> <li><code>help(print)</code></li> <li>help(dict)</li> <li><code>help('assert')</code></li> <li><code>import math; help(math.log)</code></li> </ul></li> <li>Also returns docstrings from your non-built-in stuff, like your own methods.</li> </ul> <p><strong>Michael #6:</strong> <strong>Python Architecture Graphs</strong></p> <ul> <li>by David Seddon</li> <li><a href="">Impulse</a> - a CLI which allows you to quickly see a picture of the import graph any installed Python package at any level within the package.</li> <li>Useful to run on an unfamiliar part of a code base, to help get a quick idea of the structure. </li> <li>It's a visual explorer to give you a quick signal on architecture.</li> <li><a href="">Import Linter</a> - this allows you to declare and check contracts about your dependency graph, which gives you the ability to lint your code base against architectural rules. </li> <li>Helpful to enforce certain architectural constraints and prevent circular dependencies creeping in.</li> </ul> <p><strong>Extras</strong></p> <p>Michael:</p> <ul> <li><a href="">tabnanny</a></li> <li>flask course is out, give it <a href="">a look</a></li> </ul> <p><strong>Jokes</strong> </p> <p>Two threads walk into a bar. The barkeeper looks up and yells, 'Hey, I want don't any conditions race like time last!’</p> <p>A string value walked into a bar, and then was sent to stdout.</p>
Jul 23, 2019
#139 f"Yes!" for the f-strings
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>Special guest: <a href=""><strong>Ines Montani</strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Simplify Your Python Developer Environment</strong></a></p> <ul> <li>Contributed by Nils de Bruin</li> <li>“Three tools (pyenv, pipx, pipenv) make for smooth, isolated, reproducible Python developer and production environments.”</li> <li>The tools: <ul> <li>pyenv - install and manage multiple Python versions and flavors</li> <li>pipx - install a Python application with it’s own virtual environment for use globally</li> <li>pipenv - managing virtual environments, dependencies, on a per project basis</li> </ul></li> <li>Brian note: I’m not sold on any of these yet, but honestly haven’t given them a fair shake either, but also didn’t really know how to try them all out. This is a really good write up to get started. </li> </ul> <p><strong>Ines #2:</strong> <a href=""><strong>New course: A Code-First Introduction to Natural Language Processing</strong></a></p> <ul> <li><a href=""></a> is a really popular, free course for deep learning by Rachel Thomas and Jeremy Howard</li> <li>Also comes with a Python library and lots of notebooks</li> <li>Some influential research developed alongside the course, e.g. <a href="">ULMFiT</a> (popular algorithm for NLP tasks like text classification)</li> <li>New course on Natural Language Processing: <ul> <li>Practical introduction to NLP covering both modern neural network approaches <em>and</em> traditional techniques</li> <li>Highlights: <ul> <li>NLP background: topic modeling and linear models</li> <li>Rule-based approaches and real-world problem solving</li> <li>Focus on ethics – videos on bias and disinformation</li> </ul></li> </ul></li> </ul> <p><strong>Michael #3:</strong> <a href=""><strong>Cloning the human voice</strong></a></p> <ul> <li>In 5 minutes, with Python</li> <li>via Brenden</li> <li>Clone a voice in 5 seconds to generate arbitrary speech in real-time</li> <li>An implementation of <a href="">Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis</a> (SV2TTS) with a vocoder that works in real-time.</li> <li>Watch the video: <strong><a href=""></a></strong> </li> <li>Also: <a href="">Fake voices 'help cyber-crooks steal cash</a>’</li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>Ab(using) pyproject.toml and stuffing pytest.ini and mypy.ini content into it</strong></a></p> <ul> <li>Contributed by Andrew Spittlemeister </li> <li>My first reaction is horror, but this is kinda my thought process with this one <ul> <li>toml is not ini (but they look close)</li> <li>neither pytest nor mypy support storing configuration in pyproject.toml</li> <li>they both do support using setup.cfg (but flit and poetry projects don’t use that file, or try not to)</li> <li>they both support passing in the config file as a command line argument</li> <li>you can be careful and write a pyproject.toml file that is both toml and ini compliant</li> <li>drat, this is a reasonable idea, if not a little wacky</li> <li>no <strong>guarantee</strong> that it will keep working</li> </ul></li> <li>one thing to note: use quotes for stuff you normally wouldn’t need to in ini file.</li> </ul> <p>Example ini: </p> <pre><code> [pytest] addopts = -ra -v </code></pre> <p>if stuffed in pyproject.toml</p> <pre><code> [pytest] addopts = "-ra -v" </code></pre> <p>to run:</p> <pre><code> &gt; mypy --config-file pyproject.toml module_name &gt; pytest -c pyproject.toml </code></pre> <p><strong>Ines #5:</strong> <strong><em>*<a href=""></strong>Polyaxon</em>*</a></p> <ul> <li><em>A platform for reproducing and managing the whole life cycle of machine learning and deep learning applications.</em></li> <li>We talked to lots of research groups and everyone works with just their GPU on desktop. Super slow – you need to wait for results, schedule next job etc.</li> <li>Polyaxon is a free open source library built on Kubernetes. Really easy to set up, especially on Google Kubernetes Engine.</li> <li>Especially good for hyper-parameter search, where you might not need GPU experiments if you can run lots of experiments in parallel</li> <li><a href="">Release v0.5</a> just came today. Big improvements: <ul> <li>Plugins system</li> <li>Local runs, for much easier debugging</li> <li>New workflow engine for chaining things together and run experiments with lots of steps</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Flynt for f-strings</strong></a></p> <ul> <li>A tool to automatically convert old string literal formatting to f-strings</li> <li>F-Strings: Not only are they more readable, more concise, and less prone to error than other ways of formatting, but they are also faster!</li> <li>Converted over 500 lines / expressions in Talk Python Training and Python Bytes.</li> <li>Get started with a <a href=""><strong>pipx</strong></a> install: <code>pipx install flynt</code></li> <li>Then point it at <ul> <li>A file: <code>flynt</code></li> <li>A directory (recursively): <code>flynt ./</code></li> </ul></li> <li>Converts code like this: <code>print(``"``Greetings {}, you have found {:,} items!``"``.format(name, count))</code></li> <li>To code like this: <code>print(f"Greetings {name}, you have found {count:,} items!")</code></li> <li>Beware of the digit grouping bug.</li> <li>Good project to jumping in and contributing to open source</li> </ul> <p>Extras:</p> <ul> <li>Thanks to André Jaenisch for pointing the existence of <a href="">ReDoS attacks and a good video explaining them</a>.</li> </ul> <p>Michael:</p> <ul> <li><a href="">Python httptoolkit</a></li> <li><a href="">Python Magic’s name</a> via David Martínez</li> <li>Flying Fractals (<a href="">video</a> and <a href="">code</a>)</li> <li>Python 3.7.4 is out</li> </ul> <p>Ines:</p> <ul> <li><a href="">Explosion</a> <a href="">(?)</a></li> <li><a href="">spaCy IRL 2019</a> <ul> <li>our very first conference held on July 6 in Berlin</li> <li>many amazing speakers from research, applied NLP and the community</li> <li>all talks were recorded and will be up on <a href="">our YouTube channel</a> very soon</li> </ul></li> <li><a href="">FastAPI</a> core developer Sebastián Ramírez is joining our team <ul> <li>FastAPI was <a href="">presented by Brian</a> in episode 123 of this podcast</li> <li>we’re big fans and have been switching all our APIs over to FastAPI</li> <li>we’ll keep supporting the project and will definitely give Sebastián enough time to keep working on it</li> </ul></li> </ul> <p>Joke:</p> <ul> <li>A programmer walks into a bar and orders 1.38 root beers. The bartender informs her it's a root beer float. She says 'Make it a double!’</li> <li>What do you call a developer without a side project? <ul> <li>Well rested.</li> </ul></li> </ul>
Jul 18, 2019
#138 Will PyOxidizer weld shut one of Python's major gaps?
<p>Sponsored by <strong>DigitalOcean:</strong> <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>flake8-comprehensions</strong></a></p> <ul> <li>submitted by Florian Dahlitz</li> <li>I’m already using flake8, so adding this plugin is a nice idea.</li> <li>checks your code for some generator and comprehension questionable code. <ul> <li>C400 Unnecessary generator - rewrite as a list comprehension.</li> <li>C401 Unnecessary generator - rewrite as a set comprehension.</li> <li>C402 Unnecessary generator - rewrite as a dict comprehension.</li> <li>C403 Unnecessary list comprehension - rewrite as a set comprehension.</li> <li>C404 Unnecessary list comprehension - rewrite as a dict comprehension.</li> <li>C405 Unnecessary (list/tuple) literal - rewrite as a set literal.</li> <li>C406 Unnecessary (list/tuple) literal - rewrite as a dict literal.</li> <li>C407 Unnecessary list comprehension - '[HTML_REMOVED]' can take a generator.</li> <li>C408 Unnecessary (dict/list/tuple) call - rewrite as a literal.</li> <li>C409 Unnecessary (list/tuple) passed to tuple() - (remove the outer call to tuple()/rewrite as a tuple literal).</li> <li>C410 Unnecessary (list/tuple) passed to list() - (remove the outer call to list()/rewrite as a list literal).</li> <li>C411 Unnecessary list call - remove the outer call to list().</li> </ul></li> <li>Example: <ul> <li>Rewrite <code>list(f(x) for x in foo)</code> as <code>[f(x) for x in foo]</code></li> <li>Rewrite <code>set(f(x) for x in foo)</code> as <code>{f(x) for x in foo}</code></li> <li>Rewrite <code>dict((x, f(x)) for x in foo)</code> as <code>{x: f(x) for x in foo}</code></li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>PyOxidizer (again)</strong></a></p> <ul> <li>Michael’s assessment - There are three large and looming threats to Python. Lack of <ul> <li>A real mobile development story</li> <li>GUI applications on desktop operating systems</li> <li>Sharing your application with users (this is VERY far from deployment to servers)</li> </ul></li> <li>Cover PyOxidizer before but seems to have just rocketed off last couple of weeks.</li> <li>At their PyCon 2019 keynote talk, Russel Keith-Magee <a href="">identified code distribution</a> as a potential <em>black swan</em> - an existential threat for longevity - for Python. <ul> <li>“<em>Python hasn't ever had a consistent story for how I give my code to someone else, especially if that someone else isn't a developer and just wants to use my application.</em>”</li> </ul></li> <li>They announced the first release of PyOxidizer (<a href="">project</a>, <a href="">documentation</a>), an open source utility that aims to solve the Python application distribution problem!</li> <li><strong>PyOxidizer's marquee feature is that it can produce a single file executable containing a fully-featured Python interpreter, its extensions, standard library, and your application's modules and resources.</strong> </li> <li>You can have a single <code>.exe</code> providing your application. </li> <li>Unlike other tools in this space which tend to be operating system specific, PyOxidizer works across platforms (currently Windows, macOS, and Linux - the most popular platforms for Python today).</li> <li><strong>PyOxidizer loads everything from memory and there is no explicit I/O being performed. When you</strong> <code>**import**</code> <strong>a Python module, the bytecode for that module is being loaded from a memory address in the executable using zero-copy.</strong></li> <li>This makes PyOxidizer executables <a href="">faster</a> to start and <code>import</code> - faster than a <code>python</code> executable itself!</li> </ul> <p><strong>Brian #3:</strong> <strong>Using</strong> <code>changedir</code> <strong>to avoid the need for</strong> <code>src</code></p> <ul> <li>I’ve been experimenting with combining flit, pytest, tox, and coverage for new projects.</li> <li>And in doing so, ran across a cool feature of tox that I didn’t know about before, <code>changedir</code>.</li> <li>It’s a feature of tox to allow you to run tests in a different directory than the top level project directory. <ul> <li><a href="">tox changedir docs</a></li> <li><a href="">tox and pytest and changedir</a></li> </ul></li> <li>I talk about this more in <a href="">episode 80 of Test &amp; Code</a>. <ul> <li>As an example project I build yet another markdown converter using regular expressions.</li> <li>This is funny to me, considering the recent cloudflare outage due to a single regular expression.</li> <li>“Tragedy is what happens to me, comedy is what happens to you” - Mel Brooks approximate quote.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>WebRTC and ORTC implementation for Python using asyncio</strong></a></p> <ul> <li><a href="">Web Real-Time Communication (WebRTC)</a> - WebRTC is a free, open project that provides browsers and mobile applications with Real-Time Communications (RTC) capabilities via simple APIs.</li> <li><a href="">Object Real-Time Communication (ORTC)</a> - ORTC (Object Real-Time Communications) is an API allowing developers to build next generation real-time communication applications for web, mobile, or server environments.</li> <li>The API closely follows its Javascript counterpart while using pythonic constructs: <ul> <li>promises are replaced by coroutines</li> <li>events are emitted using <code>pyee.EventEmitter</code></li> </ul></li> <li>The main WebRTC and ORTC implementations are either built into web browsers, or come in the form of native code.</li> <li>In contrast, the <code>aiortc</code> implementation is fairly simple and readable. <ul> <li>Good starting point for programmers wishing to understand how WebRTC works or tinker with its internals. </li> <li>Easy to create innovative products by leveraging the extensive modules available in the Python ecosystem. </li> <li>For instance you can build a full server handling both signaling and data channels or apply computer vision algorithms to video frames using OpenCV.</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Apprise - Push Notifications that work with just about every platform!</strong></a></p> <ul> <li>listener suggestion</li> <li>cool shim project to allow multiple notification services in one app</li> <li>“<em>Apprise</em> allows you to send a notification to <em>almost</em> all of the most popular <em>notification</em> services available to us today such as: Telegram, Pushbullet, Slack, Twitter, etc. <ul> <li>One notification library to rule them all.</li> <li>A common and intuitive notification syntax.</li> <li>Supports the handling of images (to the notification services that will accept them).”</li> </ul></li> <li>supports <ul> <li>notification services such as discord, gitter, ifttt, mailgun, mattermost, MS teams, twitter, …</li> <li>SMS notification through Twilio, Nexmo, AWS, D7</li> <li>email notifications</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Websauna</strong></a> <a href=""><strong>web framework</strong></a></p> <ul> <li>Websauna is a full stack Python web framework for building web services and back offices with admin interface and sign up process</li> <li>"<em>We have web applications 80% figured out. Websauna takes it up to 95%.</em>”</li> <li>Built upon Python 3, Pyramid, and SQLAlchemy.</li> <li>When to use it? <ul> <li>Websauna is focused on Internet facing sites where you have a public or private sign up process and an administrative interface. Its sweet spots include custom business portals and software-as-a-service products which are too specialized for off-the-shelf solutions.</li> </ul></li> <li>Benefits <ul> <li>Focus on core business logic as Websauna provides basic website building blocks like sign up and sign in. </li> <li>Low learning curve and friendly comprehensive documentation help novice developers</li> <li>Emphasis is on meeting business requirements with reliable delivery times, responsiveness, consistency</li> <li>Site operations is half the story. Websauna provides an automated deployment process and integrates with monitoring, security and other DevOps solutions.</li> </ul></li> </ul> <p><strong>Extras</strong></p> <p>Michael:</p> <ul> <li><a href=""><strong>Data driven Flask course</strong></a> is out!</li> </ul> <p>Brian:</p> <ul> <li>Recent <a href="">Test &amp; Code episodes</a> were solo because I’m in the middle of a work move and didn’t want to schedule interviews around a crazy work schedule. However, that should settle down in July and I can get back to getting great guests on the show. But I’m also having fun with solo topics, so I’ll keep that in the mix. <ul> <li>upshot: if I’ve contacted you or you me about being on the show and you haven’t heard from me lately, give me a nudge with a DM or email or something.</li> </ul></li> </ul> <p><strong>Jokes</strong> </p> <ul> <li>An SQL query goes into a bar, walks up to two tables and asks, 'Can I join you?'</li> <li>Not a joke, really, but along the lines of “comedy when it happens to you”. <ul> <li>Reset procedure for GE lightbulbs <a href=""></a></li> </ul></li> </ul>
Jul 08, 2019
#137 Advanced Python testing and big-time diffs
<p>Sponsored by Rollbar: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Comparing the Same Project in Rust, Haskell, C++, Python, Scala and OCaml</strong></a></p> <ul> <li>Tristan Hume, writing about a university project</li> <li>Teams of up to 3 people, multi month, write a Java to x86 compiler in language of choice</li> <li>Needed to pass both known and unknown tests.</li> <li>Secret tests to be run after submission encouraged teams to add more testing than provided.</li> <li>Nothing but standard libraries, and no parsing libraries, even if in standard.</li> <li>Lines of code <ul> <li>Rust baseline</li> <li>Haskell: 1-1.6x </li> <li>C++: 1.4x</li> <li>Rust (another team): 3x</li> <li>Scala: 0.7 x</li> <li>OCaml: 1-1.6x</li> <li>Python: about half the size</li> </ul></li> <li>Python version <ul> <li>one person</li> <li>used metaprogramming</li> <li>more extra features than any other team</li> <li>passed all public and secret tests</li> </ul></li> </ul> <p><strong>Michael #2 :</strong> <a href=""><strong>Pylustrator is a program to style your matplotlib plots</strong></a></p> <ul> <li>via Len Wanger</li> <li>Pylustrator is a program to style your matplotlib plots for publication. </li> <li>Subplots can be resized and dragged around by the mouse, text and annotations can be added. </li> <li>Changes can be saved to the initial plot file as python code.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>MongoDB 4.2</strong></a> </p> <ul> <li>Distributed Transactions <ul> <li>extends multi-document ACID transactions across documents, collections, dbs in a replica set, and sharded cluster.</li> </ul></li> <li>Field Level Encryption <ul> <li>encryption done on client side</li> <li>satisfies GDPR by allowing customer key destruction rendering server data on customer useless.</li> <li>system administration can be done with no exposure to private data</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Deep Difference and search of any Python object/data</strong></a></p> <ul> <li>via <a href="">François Leblanc</a></li> <li><strong>DeepDiff</strong>: Deep Difference of dictionaries, iterables, strings and other objects. It will recursively look for all the changes.</li> <li>Lots of nice touches: <ul> <li>List difference ignoring order or duplicates</li> <li>Report repetitions</li> <li>Exclude certain types from comparison</li> <li>Exclude part of your object tree from comparison</li> <li>Significant Digits</li> </ul></li> <li><strong>DeepSearch</strong>: Search for objects within other objects.</li> <li><strong>DeepHash</strong>: Hash of ANY python object based on its contents even if the object is not considered hashable! DeepHash is supposed to be deterministic in order to make sure 2 objects that contain the same data, produce the same hash.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Advanced Python Testing</strong></a></p> <ul> <li>Josh Peak</li> <li>“This article is mostly for me to process my thoughts but also to pave a path for anyone that wants to follow a similar journey on some more advanced python testing topics.”</li> <li>Learning journey (including some great podcasts and an awesome book on testing)</li> <li>Testing tools <ul> <li>basic test structure</li> <li>adding black to testing with pytest-black</li> <li>linting with pylint <ul> <li>including a very cool speed up trick to only lint modified files.</li> </ul></li> <li>flake8, including docstring checking</li> <li>tox.ini modifications</li> <li>code coverage goals and how to ratchet up to that goal with <code>--cov-fail-under</code> <ul> <li>cool learning: “Increase code coverage by testing more code OR deleting code.”</li> </ul></li> <li>fixtures for database connections</li> <li>utilizing mocks, spies, stubs, and monkey patches, including <code>pytest-mock</code></li> <li><code>pytest-vcr</code> to save network interactions and replay them in future test runs, resulting in a 10x speedup.</li> </ul></li> <li>Lots of links and tangents possible from this article.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Understanding Python's del</strong></a></p> <ul> <li>via Kevin Buchs </li> <li><a href=""><strong>Official docs</strong></a></li> <li>General confusion of what this does</li> <li>Looks like memory management, and it mostly isn’t</li> <li>Primary use: remove an item from a list given its index instead of its value or from a dictionary given its key: <code>del person['profession'] # person is a dict</code> </li> <li>del statement can also be used to remove slices from a list <code>del lst[2:4]</code></li> <li>del can also be used to delete entire variables: <code>del variable</code></li> <li>Recently covered how <a href=""><strong>The CPython Bytecode Compiler is Dumb</strong></a>. Proactive dels could help.</li> </ul> <p><strong>Extras</strong></p> <p>Michael:</p> <ul> <li><a href=""><strong>Pynsource</strong></a>: Reverse engineer Python source code into UML diagrams (via Anders Klint)</li> <li><a href=";"><strong>Language Bar chart</strong></a> race (via <a href="">Josh Thurston</a>)</li> <li><a href=""><strong>My Local maximum appearance</strong></a>.</li> </ul> <p><strong>Jokes</strong> </p> <p>Optimist: The glass is half full. Pessimist: The glass is half empty. Programmer: The glass is twice as large as necessary.</p> <p>Pragmatist: allowing room for requirements oversights, scope creep, and schedule overrun.</p> <p>From “The Upside” with Kevin Hart and Bryan Cranston (watched it last night): K: Would you invest in [HTML_REMOVED]? B: That seems too niche. K: What’s “niche” mean? B: It’s the girl version of “nephew”.</p>
Jul 02, 2019
#136 A Python kernel rather than cleaning the batteries?
<p>Brought to you by Datadog: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Voilà!</strong></a></p> <ul> <li>“from Jupyter notebooks to standalone applications and dashboards”</li> <li>Turn a notebook into a web app with: <ul> <li>custom widgets</li> <li>runnable code (but not editable)</li> <li>interactive plots</li> <li>different custom grid layouts</li> <li>templates</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Toward a “Kernel Python”</strong></a></p> <ul> <li>By Glyph</li> <li>Glyph wants to Marie Kondō the standard library (and I think I agree with him)</li> <li>We have <a href=""><strong>PEP 594</strong></a> for removing obviously obsolete and unmaintained detritus from the standard library.</li> <li>PEP 594 is great news for Python, and in particular for the maintainers of its standard library, who can now address a reduced surface area.</li> <li>Believes the PEP may be approaching the problem from the wrong direction.</li> <li>One “dead” battery is the <code>colorsys</code> module: why not remove it? “The module is useful to convert CSS colors between coordinate systems. Today, however, the modules you need to convert colors between coordinate systems are only a <code>pip install</code> away.</li> <li>Every little bit is overhead for the core devs, consider the state of PRs</li> <li>Looking at <a href=" is%3Aopen label%3A%22awaiting review%22 sort%3Aupdated-asc">CPython’s keyword-based review queue</a>, we can see that there are <em>429</em> tickets currently awaiting review. The oldest PR awaiting review hasn’t been touched since February 2, 2018, which is almost 500 days old.</li> <li>By Glyph’s subjective assessment, on this page of 25 PRs, 14 were about the standard library, 10 were about the core language or interpreter code</li> <li>We need a “kernel” version of Python that contains only the most absolutely minimal library, so that all implementations can agree on a core baseline that gives you a “python”</li> <li>Michael: There will be a cost to beginners. But there is already.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Use</strong></a></p> <ul> <li>I didn’t know it was that easy to get <code>python -m [HTML_REMOVED]</code> to work.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>The CPython Bytecode Compiler is Dumb</strong></a></p> <ul> <li>by Chris Wellons</li> <li>Given multiple ways to express the same algorithm or idea, Chris tends to prefer the one that compiles to the more efficient bytecode.</li> <li>Fortunately CPython, the main and most widely used implementation of Python, is very transparent about its bytecode. It’s easy to inspect and reason about its bytecode. The disassembly listing is easy to read and understand.</li> <li>One fact has become quite apparent: <strong>the CPython bytecode compiler is pretty dumb</strong>. With a few exceptions, it’s a very literal translation of a Python program, and there is almost <a href="">no optimization</a>.</li> <li><a href="">Darius Bacon points out</a> that Guido van Rossum himself said, “<a href=";pg=PA26&amp;lpg=PA26&amp;dq=%22Python is about having the simplest, dumbest compiler imaginable.%22&amp;source=bl&amp;ots=2OfDoWX321&amp;sig=ACfU3U32jKZBE3VkJ0gvkKbxRRgD0bnoRg&amp;hl=en&amp;sa=X&amp;ved=2ahUKEwjZ1quO89bgAhWpm-AKHfckAxUQ6AEwAHoECAkQAQ#v=onepage&amp;q=%22Python%20is%20about%20having%20the%20simplest%2C%20dumbest%20compiler%20imaginable.%22&amp;f=false">Python is about having the simplest, dumbest compiler imaginable.</a>” So this is all very much by design.</li> <li>The consensus seems to be that if you want or need better performance, use something other than Python. (And if you can’t do that, at least use PyPy.) ← Cython people, Cython.</li> <li>Example</li> </ul> <pre><code> def foo(): x = 0 y = 1 return x </code></pre> <p>Could easily be:</p> <pre><code> def foo(): return 0 </code></pre> <p>Yet, CPython completely misses this optimization for both x and y:</p> <pre><code> 2 0 LOAD_CONST 1 (0) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (1) 6 STORE_FAST 1 (y) 4 8 LOAD_FAST 0 (x) 10 RETURN_VALUE </code></pre> <p>And so on.</p> <ul> <li>Brett Cannot has expressed performance as a major focus for CPython, maybe there is something here?</li> </ul> <p><strong>Brian #5:</strong> <strong>You can play with EdgeDB now, maybe</strong></p> <ul> <li><a href="">A Path to a 10x Database</a></li> <li><a href="">EdgeDB roadmap</a></li> <li>Alpha 1 is available. </li> <li>“EdgeDB is the next generation relational database based on PostgreSQL. It features a novel data model and an advanced query language.”</li> <li>I’m excited about what their doing. Looking forward to 1.0.</li> <li>Lots of great features listed in the 10x post, but what I’m most intrigued by is their <a href="">replacement of SQL with a different query language</a>.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>16 Python libraries that helped a healthcare startup grow</strong></a></p> <ul> <li>via <a href="">Waqas Younas</a></li> <li>Worked with a U.S.-based healthcare startup for 7 years. This startup developed a software product that sent appointment reminders to the patients of healthcare facilities; the reminders were sent via email, text, and IVR.</li> </ul> <ol> <li>Paramiko - A Python implementation of SSHv2.</li> <li>built-in CSV module</li> <li>SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper</li> <li>Requests - HTTP for Humans™</li> <li>BeautifulSoup - Python library for pulling data out of HTML and XML files.</li> <li>testscenarios - a pyunit extension for dependency injection</li> <li>HL7 - a simple library for parsing messages of Health Level 7 (HL7) version 2.x into Python objects. </li> <li>Python-Phonenumbers - Library for parsing, formatting, and validating international phone numbers</li> <li>gevent - a coroutine -based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libev or libuv event loop.</li> <li>dateutil - powerful extensions to datetime (pip install python-dateutil)</li> <li>Matplotlib - a Python 2D plotting library which produces publication quality figures</li> <li>python-magic - a python interface to the libmagic file type identification library. libmagic identifies file types by checking their headers according to a predefined list of file types.</li> <li>Django - a high-level Python Web framework that encourages rapid development and clean, pragmatic design</li> <li>Boto - a Python package that provides interfaces to Amazon Web Services.</li> <li>Mailgun Python bindings - helped us send appointment reminders seamlessly</li> <li>Twilio’s Python bindings - helped us send appointment reminders seamlessly</li> </ol> <p><strong>Extras</strong></p> <p>Michael:</p> <p><a href="">United States Digital Service</a></p> <p><strong>Jokes</strong> </p> <p>Difference between ML &amp; AI? <a href=""><strong>Ans</strong></a>.</p>
Jun 25, 2019
#135 macOS deprecates Python 2, will stop shipping it (eventually)
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>Special guest <a href=""><strong>Max</strong></a> <a href=""><strong>Sklar</strong></a> </p> <p><strong>Brian #1:</strong> <a href=""><strong>Why do Python lists let you += a tuple, when you can’t + a tuple?</strong></a></p> <ul> <li>Reuven Lerner</li> </ul> <pre><code> &gt;&gt;&gt; x = [1, 2, 3] &gt;&gt;&gt; b = (4, 5, 6) &gt;&gt;&gt; x + b Traceback (most recent call last): File "[HTML_REMOVED]", line 1, in [HTML_REMOVED] TypeError: can only concatenate list (not "tuple") to list &gt;&gt;&gt; x += b &gt;&gt;&gt; x [1, 2, 3, 4, 5, 6] </code></pre> <ul> <li>Huh??</li> <li>“It turns out that the implementation of <code>list.__iadd__</code> (in place add) takes the second (right-hand side) argument and adds it, one element at a time, to the list. It does this internally, so that you don’t need to execute any assignment after. The second argument to “+=” must be iterable.”</li> </ul> <p><strong>Max #2</strong>: <a href=""><strong>R vs Python, R is out of top 20 languages despite statistical boom</strong></a></p> <ul> <li>Subtitle: is R declining because of Python?</li> <li>First of all, this article is about an index on the popularity of programming languages from an organization TIOBE. They have an index on the popularity of programming languages. Obviously it’s a combination of many different scores, and that could be controversial, but I’m going to assume that they put some thought into how the rankings are calculated, and that it’s as good as any.</li> <li>A few stories here: first Python hit at all time high in their ranking at number 3, beating out c++ I believe for the first time, and only Java and C are above it.</li> <li>The other story is that the statistical language R dipped below 20 to number 21, and the speculation is that Python has sort of taken over as the preferred statistical language to R.</li> <li>Personally, I got into Python much sooner, because I started as a software engineer, and moved into data science and machine learning. So after taking CS, and programming in Java and C for a few years, python came much more naturally.</li> <li>But still - a lot of people who are data-science first (and they have an additional skills to the kind of hybrid that I am) like and prefer R, and they can use it in a specialized way and get good results.</li> <li>Personally, I’m going to stick with python, because there’s so many statistical libraries yet to learn, and it’s served me well thus far.</li> <li>The language I’ve used most in recent years, Scala, is surprisingly down at 31 - not even close!</li> <li>related: <strong><a href=""></a></strong></li> </ul> <p><strong>Michael #3:</strong> <a href=""><strong>macOS deprecates Python 2, will stop shipping it (eventually)</strong></a></p> <ul> <li>via Dan Bader, on the heels of WWDC 2019</li> <li>“Future versions of macOS won’t include scripting language runtimes by default”</li> <li>Contrast this with Windows just now starting to ship with Python 3</li> <li>In the same announcement: </li> <li>“Use of Python 2.7 isn’t recommended as this version is included in macOS for compatibility with legacy software. Future versions of macOS won’t include Python 2.7. Instead, it’s recommended that you run <code>python3</code> from within Terminal. (51097165)”</li> <li>Also has impact wider than “us”. E.g. No Ruby or Perl, means home brew doesn’t install easily which is how we get Python 3!</li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>Pythonic Ways to Use Dictionaries</strong></a></p> <ul> <li>Al Sweigart</li> <li>A few pythonic uses of dictionaries that are not obvious to new people.</li> <li>Use <code>get()</code> and <code>setdefault()</code> with Dictionaries <ul> <li><code>get(key, default=[HTML_REMOVED])</code> allows you to read a key without checking for it’s existence beforehand.</li> <li><code>setdefault(key, default=[HTML_REMOVED])</code> is a bit of a strange duck but still useful. Set the value of something if it doesn’t exist yet.</li> </ul></li> <li>Python Uses Dictionaries Instead of a Switch Statement <ul> <li>Just do it a few times to get the hang of it. Then it becomes natural.</li> </ul></li> <li>Michael's switch addition for Python: <a href=""><strong></strong></a> </li> </ul> <p><strong>Max #5:</strong> <a href=""><strong>Things you are probably not using in Python 3 But Should</strong></a></p> <ul> <li>This is from</li> <li>This is particularly relevant for me, since I used python legacy at Foursquare for many years, and now coming back to it taking another look at python v3.</li> <li>One that looks very useful is f-Strings where you can put the variable name in braces in a string and just have it replaced. I’ve seen things like this in other languages - notably PHP and most front-end scripts. Makes the code very readable. <ul> <li>Except I know I’m going to screw up by leaving out that stray “f” in front of the string. It should almost be automatic, because how often are you putting these variable names in braces?</li> </ul></li> <li>Another thing I didn’t know python 3 had - again I’m kind of just get started with python 3 is enumerations.</li> <li>I’ve been using Enums for years in scala (really case classes) to make my code WAY more readable. Will keep that in mind when developing in python 3.</li> </ul> <p><strong>Michael #6:</strong> <strong>Have a time machine? C++ would get the Python 2 → 3 treatment too</strong></p> <ul> <li>via James Small</li> <li>In <a href=""><strong>a recent CppCast interview</strong></a>, Herb Sutter describes how he would change C/C++ types if he could go back in time.</li> <li>This is almost exactly how things were changed from Python 2 to Python 3 (str split into Unicode strings and byte arrays)</li> <li>So my question to you two is: Why was the transition so hard? Was it just habit and stubbornness? What could the PSF have done?</li> </ul> <p><strong>Extras</strong></p> <p>Michael:</p> <ul> <li><a href=""><strong>pip install mystery</strong></a> <ul> <li>by Divo Kaplan</li> <li>A random Python package every time.</li> <li>Mystery is a Python package that is instantiated as a <em>different</em> package every time you install it!</li> <li>Inspired by one of our episodes</li> </ul></li> <li>Get our effective pycharm book bundle with the courses over at <a href=""><strong></strong></a></li> </ul> <p>Brian:</p> <ul> <li><a href=""><strong>Python 3.8.0b1</strong></a> <ul> <li>If you support a package, please test.</li> </ul></li> </ul> <p>Max: </p> <ul> <li><a href=""><strong>The Local Maximum</strong></a> <ul> <li>Weekly Podcast that covers both the theoretical issues in probability theory, philosophy, and machine learning, but then applies it in a practical way to things like current events and product development.</li> <li>For example, a few weeks ago I did a show on how to estimate the probably of an event that has never occurred</li> <li>We also cover things like Apple’s decision to breakup iTunes, how the internet is shaping up in places like Cuba, and the controversy around YouTube’s recommendation algorithm.</li> </ul></li> </ul> <p><strong>Jokes</strong> </p> <p>MK: There are only two hard problems in Computer Science: cache invalidation, naming things and off-by-one-errors.</p>
Jun 20, 2019
#134 Python proves Mercury is the closest planet to Earth
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Three scientists publish a paper proving that Mercury, not Venus, is the closest planet to Earth.</strong></a> <strong>using Python</strong></p> <ul> <li><p>contributed by, and explained by, listener Andrew Diederich.</p> <p>“This is from the March 19th, 2019 Strange Maps article. Which planet is, on average, closest to the Earth? Answer: Mercury. Actually, Mercury is, on average, the closest to all other planets, because it’s closest to the sun.”</p></li> <li><p>article, including video, uses <a href="">PyEphem</a>, which apparently is now deprecated and largely replaced with <a href="">skyfield</a>. </p></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Github semantics</strong></a></p> <ul> <li>Parsing, analyzing, and comparing source code across many languages</li> <li>Written in a Haskell, it’s a library and command line tool for parsing, analyzing, and comparing source code.</li> <li>It’s still early days yet, but semantic can do a lot of cool things, and is powering public-facing GitHub features. I’m tremendously excited as to see how it’ll evolve now that it’s a community-facing project.</li> <li>Understands: Python, TypeScript, JavaScript, Ruby, Go, …</li> <li>here are some cool things inside it: <ul> <li>A flow-sensitive, caching, generalized interpreter for imperative languages</li> <li>An abstract interpreter that generates scope graphs for a given program text</li> <li>A strategic rewriting system based on recursion schemes for open syntax terms</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>flake8-black</strong></a></p> <ul> <li>Contributed by Nathan Clayton</li> <li>“The point of this plugin is to be able to run <code>black --check ...</code> from within the flake8 plugin ecosystem.”</li> <li>I like to run flake8 during development both to keep things neat, and to train myself to just write code in a more standard way. This is a way to run <code>black</code> with no surprises.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Python Preview</strong></a> <a href=""><strong>for VS Code</strong></a></p> <ul> <li>You write Python code (script style mostly), it creates an object-visualization</li> <li>Think of a picture your first year C++ CS prof might draw. This extension does that automatically as you write Python code</li> <li>Looks to be based (conceptually) on <a href="">Philip Guo</a>’s <a href="">Python Tutor</a> site.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Create and Publish a Python Package with Poetry</strong></a></p> <ul> <li>John Franey</li> <li>Walks through creating a package, customizing the <code>pyproject.toml</code>, and talks about the different settings in the toml and what it means.</li> <li>Then using the testpypi, and finally publish.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Pointers in Python: What's the Point?</strong></a></p> <ul> <li>by Logan Jones</li> <li>Quick question: Does Python have pointers (outside of C-extensions, etc of course)?</li> <li>Yet Python is more pointer heavy than most languages (more so than C# more so than even C++)!</li> <li>In Python, everything is an object, even numbers and booleans.</li> <li>Each object contains at least three pieces of data: <ul> <li>Reference count</li> <li>Type</li> <li>Value</li> </ul></li> <li>Check that you have the same object <code>is</code> instead of <code>==</code></li> <li>Python variables are pointers, just safe ones.</li> <li>Interesting little tidbit from the article: <ul> <li>Interning strings is useful to gain a little performance on dictionary lookup—if the keys in a dictionary are interned, and the lookup key is interned, the <strong>key comparisons</strong> (after hashing) <strong>can be done by a pointer compare</strong> instead of a string compare. (Source)</li> </ul></li> <li>But like we have inline-assembly in C++ and unsafe mode in C#, we <a href="">can use pointers in Cython</a> or more fine-grained with <a href="">ctypes</a>.</li> </ul> <p><strong>Extras</strong></p> <p>Michael:</p> <ul> <li>PSF needs your help. Spread the word about the fundraiser and please, ask your company to contribute: <a href="">Building the PSF: the Q2 2019 Fundraiser</a> (Donations are tax-deductible for individuals and organizations that pay taxes in the United States) <ul> <li>“Contributions help fund workshops, conferences, pay meetup fees, support fiscal sponsorships, PyCon financial aid, and development sprints. ”</li> </ul></li> </ul> <p><strong>Jokes</strong> </p> <p>via Jay Miller</p> <p>What did the developer name his newborn boy? <strong>JSON</strong></p>
Jun 12, 2019
#133 Github sponsors - The model open source has been waiting for?
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Python built-ins worth learning</strong></a></p> <ul> <li>Trey Hunner</li> <li>“I estimate <strong>most Python developers will only ever need about 30 built-in functions</strong>, but which 30 depends on what you’re actually doing with Python.” </li> <li>“I recommend triaging your knowledge: <ul> <li>Things I should memorize such that I know them well</li> <li>Things I should know <em>about</em> so I can look them up more effectively later</li> <li>Things I shouldn’t bother with at all until/unless I need them one day”</li> </ul></li> <li>all 69 built-in functions, split into <ul> <li>commonly known</li> <li>overlooked by beginners</li> <li>learn it later</li> <li>maybe learn it eventually</li> <li>you likely don’t need these</li> </ul></li> <li>Highlighting some: <ul> <li>overlooked by beginners <ul> <li>sum, enumerate, zip, bool, reversed, sorted, min, max, any, all</li> </ul></li> <li>know it’s there, but learn it later: <ul> <li>open, input, repr, super, property, issubclass, isinstance, hasattr, getattr, setattr, delattr, classmethod, staticmethod, next</li> </ul></li> </ul></li> <li>my notes <ul> <li>I think getattr should be learned early on, because it’s default behavior is so useful. But can’t use it for dicts. Use <code>mydict.get(key, default)</code> for dictionaries.</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Github sponsors and match</strong></a></p> <ul> <li>Like Patreon but for GitHub projects</li> <li>2x your sponsorship: Github matches! To boost community funding, we'll match contributions up to $5,000 during a developer’s first year in GitHub Sponsors with the GitHub Sponsors Matching Fund.</li> <li>100% to developers, Zero fees: GitHub will not charge fees for GitHub Sponsors.</li> <li>Anyone who contributes to open source—whether through code, documentation, leadership, mentorship, design, or beyond—is eligible for sponsorship.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Build a REST API in 30 minutes with Django REST Framework</strong></a></p> <ul> <li>Bennett Garner</li> <li>Very fast intro including: <ul> <li>Set up Django</li> <li>Create a model in the database that the Django ORM will manage</li> <li>Set up the Django REST Framework</li> <li>Serialize the model from step 2</li> <li>Create the URI endpoints to view the serialized data</li> </ul></li> <li>Example is a simple hero db with hero name and alias.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Dependabot has been acquired by GitHub</strong></a></p> <ul> <li>Automated dependency updates: Dependabot creates pull requests to keep your dependencies secure and up-to-date.</li> <li>I personally use and recommend PyUP: <a href=""><strong></strong></a></li> <li>How it works: <ul> <li>Dependabot checks for updates: Dependabot pulls down your dependency files and looks for any outdated or insecure requirements.</li> <li>Dependabot opens pull requests: If any of your dependencies are out-of-date, Dependabot opens individual pull requests to update each one.</li> <li>You review and merge: You check that your tests pass, scan the included changelog and release notes, then hit merge with confidence.</li> </ul></li> <li>Here's what you need to know: <ul> <li>We're integrating Dependabot directly into GitHub, starting with security fix PRs 👮‍♂️</li> <li>You can still install Dependabot from the GitHub Marketplace whilst we integrate it into GitHub, but it's now free of charge 🎁</li> <li>We've doubled the size of Dependabot's team; expect lots of great improvements over the coming months 👩‍💻👨‍💻👩‍💻👨‍💻👩‍💻👨‍💻</li> </ul></li> <li>Paid accounts are now free, automatically.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>spoof “</strong></a><a href=""><strong>New features planned for Python 4.0</strong></a><a href=""><strong>”</strong></a></p> <ul> <li>Charles Leifer - also known for Peewee ORM</li> <li>This is funny, but painful. Is it too soon to joke about the pain of 2 to 3?</li> <li>A few of my favorites <ul> <li>PEP8 will be updated. Line lengths will be increased to 89.5 characters. (compromise between 79 and 100)</li> <li>All new libraries and standard lib modules must include the phrase "for humans" somewhere in their title.</li> <li>Type-hinting has been extended to provide even fewer tangible benefits and will be called <em>type whispering</em>.</li> <li>You can make stuff go faster by adding async before every other keyword.</li> <li>Notable items left out of 4.0 <ul> <li>Still no <code>switch</code> statement.</li> <li>No improvements to packaging.</li> </ul></li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>BlackSheep web framework</strong></a></p> <ul> <li>Fast HTTP Server/Client microframework for Python asyncio, using Cython, uvloop, and httptools.</li> <li>Very Flask-like API. Interesting to consider the “popularity” of Flask vs Django in this context.</li> <li>Objectives <ul> <li>Clean architecture and source code, following <a href="">SOLID principles</a></li> <li>Intelligible and easy to learn API, similar to those of many Python web frameworks</li> <li>Keep the core package minimal and focused, as much as possible, on features defined in HTTP and HTML standards</li> <li>Targeting stateless applications to be deployed in the cloud</li> <li><a href="">High performance, see results from TechEmpower benchmarks (links in Wiki page)</a></li> </ul></li> <li>Also has an async client much like <strong>aiohttp</strong>.</li> </ul> <p><strong>Extras</strong></p> <p>Michael: </p> <ul> <li>Free courses in the <a href=""><strong>Training mobile apps</strong></a></li> <li>Upcoming webcast: <a href=""><strong>10 Tools and Techniques Python Web Developers Should Explore</strong></a></li> <li>2019 <a href=""><strong>PSF Board Elections</strong></a></li> <li>Get <a href=""><strong>PyCharm, Support Python</strong></a> <ul> <li>Until June 1st, get PyCharm at 30% OFF All the money raised will go toward the Python Software Foundation </li> </ul></li> </ul> <p><strong>Jokes</strong> </p> <ul> <li>How do you generate a random string? Put a first year Computer Science student in Vim and ask them to save and exit.</li> <li>Waiter: He's choking! Is anyone a doctor? Programmer: I'm a Vim user.</li> </ul>
Jun 05, 2019
#132 Algorithms as objects
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>History of CircuitPython</strong></a></p> <ul> <li>PSF blog, A. Jesse Jiryu Davis</li> <li>Adafruit hired Scott Shawcroft to port MicroPython to their SAMD21 chip they use on many of their boards.</li> <li>CircuitPython is a friendly fork of MicroPython. Same licensing, and they share improvements back and forth.</li> <li>“MicroPython customizes its hardware APIs for each chip family to provide speed and flexibility for hardware experts. Adafruit’s audience, however, is first-time coders. Shawcroft said, “Our goal is to focus on the first five minutes someone has ever coded.” “</li> <li>“Shawcroft aims to remove all roadblocks for beginners to be productive with CircuitPython. As he demonstrated, CircuitPython auto-reloads and runs code when the user saves it; there are two more user experience improvements in the latest release. First, serial output is shown on a connected display, so a program like <code>print("hello world")</code> will have visible output even before the coder learns how to control LEDs or other observable effects.”</li> <li>Related: <a href="">CircuitPython 4.0.0 released</a> </li> </ul> <p><strong>Michael</strong> <strong>#2</strong>: <a href=""><strong>R Risks Python Swallowing It Whole: TIOBE</strong></a></p> <ul> <li>Is the R programming language in serious trouble? According to the latest update of the TIOBE Index, the answer seems to be “yes.”</li> <li>R has finally tumbled out of the top 20 languages</li> <li>“It seems that there is a consolidation going on in the statistical programming market. Python has become the big winner.”</li> <li>Briefly speculates why is Python (which ranked fourth on this month’s list) winning big in data science? My thought: Python is a <em>full spectrum language</em> with solid numerical support.</li> </ul> <p><strong>Brian#3:</strong> <a href=""><strong>The Missing Introduction To Containerization</strong></a></p> <ul> <li>Aymen El Amri</li> <li>Understanding containerization through history <ul> <li>chroot jail, 1979, allowed isolation of a root process and it’s children from the rest of the OS, but with no security restrictions.</li> <li>FreeBSD Jail, 2000, more secure, also isolating the file system.</li> <li>Linux VServer, 2001, added “security contextes” and used new OS system-level virtualization. Allows you to run multiple Linux distros on a single VPS.</li> <li>Oracle Solaris Containers, 2004, system resource controls and boundary separation provided by “zone”.</li> <li>OpenVZ, 2005, OS-level virtualization. Used by many hosting companies to isolate and sell VPSs.</li> <li>Google’s CGroups, 2007, a mechanizm to limit and isolate resource usage. Was mainlained into Linux kernel the same year.</li> <li>LXC, Linux Containers, 2008, Similar to OpenVX, but uses CGroups.</li> <li>CloudFoundry’s Warden, 2013, an API to manage environments.</li> <li>Docker, 2013, os-level virtualization</li> <li>Google’s LMCTFY (Let me contain that for you), 2014, an OSS version of Google’s container stack, providing Linux application containers. Most of this tech is being incorporated into libcontainer.</li> <li>“Everything at Google runs on containers. There are <a href="">more than 2 billion containers</a> running on Google infrastructure every week.”</li> <li>CoreOS’s rkt, 2014, an alternative to Docker.</li> </ul></li> <li>Lots of terms defined <ul> <li>VPS, Virtual Machine, System VM, Process VM, …</li> </ul></li> <li>OS Containers vs App Containers</li> <li><p>Docker is both a Container and a Platform</p></li> <li><p>This is halfway through the article, and where I got lost in an example on creating a container sort of from scratch. I think I’ll skip to a Docker tutorial now, but really appreciate the back story and mental model of containers.</p></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Algorithms as objects</strong></a></p> <ul> <li>We usually think of an algorithm as a single function with inputs and outputs. </li> <li>Our algorithms textbooks reinforce this notion. </li> <li>They present very concise descriptions that neatly fit in half of a page. </li> <li>Little details add up until you’re left with a gigantic, monolithic function</li> <li>monolithic function lacks <strong>readability</strong></li> <li>the function also lacks <strong>maintainability</strong></li> <li><em>Nobody</em> wants to touch this code because it’s such a pain to get any context</li> <li>Complex code requires <strong>abstractions</strong></li> <li>How to tell if your algorithm is an object</li> <li><strong>Code smell #1. It’s too long or too deeply nested</strong></li> <li><strong>Code smell #2. Banner comments</strong></li> <li><strong>Code smell #3. Helper functions as nested closures, but it’s still too long</strong></li> <li><strong>Code smell #4. There are actual helper functions, but they shouldn’t be called by anyone else</strong></li> <li><strong>Code smell #5. You’re passing state between your helper functions</strong></li> <li>Write your algorithm as an object</li> <li>Refactoring a monolithic algorithm into a class improves <strong>readability</strong>, which is is our #1 goal.</li> <li>Lots of concrete examples in the article</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>pico-pytest</strong></a></p> <ul> <li>Oliver Bestwalter</li> <li>Super tiny implementation of pytest core. <a href="">25 lines</a></li> <li>My original hand crafted test framework was way more code than that, and not as readable.</li> <li>This is good to look at to understand the heart of what test frameworks do <ul> <li>find test code</li> <li>run it</li> <li>mark any exceptions as failures</li> </ul></li> <li>Of course, the bells and whistles added in the full implementation are super important, but this is the heart of what is happening.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>An Introduction to Cython, the Secret Python Extension with Superpowers</strong></a></p> <ul> <li>Cython is one of the best kept secrets of Python. </li> <li>It extends Python in a direction that addresses many of the shortcomings of the language and the platform, such as execution speed, GIL-free concurrency, absence of type checking and not creating an executable.</li> <li>Number of widely used packages that are written in it, such as <a href="">spaCy</a>, <a href="">uvloop</a>, and significant parts of <a href="">scikit-learn</a>, <a href="">Numpy</a> and <a href="">Pandas</a>.</li> <li>Cython makes use of the architectural organization of Python by translating (or 'transpiling', as it is now called) a Python file into the C equivalent of what the Python runtime would be doing, and compiling this into machine code.</li> <li>Can sometimes avoid Python types altogether (e.g. <code>sqrt</code> function)</li> <li>C arrays versus lists: Python collection types (list, dict, tuple and set) can be used as a type in <code>cdef</code> functions. The problem with the list structure, however, is that it leads to Python runtime interaction, and is accordingly slow</li> <li>Nice article for getting started and motivation. But I didn’t see Python type annotations in play (they are now supported)</li> </ul> <p><strong>Extras</strong></p> <p>Brian: </p> <ul> <li><a href=""><strong>The Price of the Hallway Track</strong></a> <strong>- Hynek</strong> <ul> <li>It’s lame to speak to an empty room, so go to some talks, and lean toward less known speakers. Definitely on my todo list for next year.</li> </ul></li> <li><a href=""><strong>Who put Python in the Windows 10 May 2019 Update?</strong></a> <strong>- Steve Dower</strong> <ul> <li>more back story</li> </ul></li> </ul> <p>Michael: </p> <ul> <li>Little development board to production via Crowd Supply: <a href=""><strong>The TinyPICO</strong></a> is an ESP32-based board that's, well, tiny ;) but packs a pretty significant punch...and it's been designed from day 1 to have first-class MicroPython support! via <a href="">matt_trentini</a></li> <li><a href=""><strong>PyCon 2019 Reflections</strong></a> by Automation Panda</li> <li>Python Bytes (yeah, us!) has a <a href=""><strong>Patreon page</strong></a>.</li> <li>Upcoming webcast: <a href=""><strong>10 Tools and Techniques Python Web Developers Should Explore</strong></a></li> </ul> <p><strong>Jokes</strong> </p> <ul> <li>What do you call eight hobbits? A hobbyte.</li> <li>Two bytes meet. The first byte asks, 'Are you ill?' The second byte replies, 'No, just feeling a bit off.’</li> <li>OR: What is Benoit B. Mandelbrot's middle name? Benoit B. Mandelbrot.</li> </ul>
May 30, 2019
#131 Python 3 has issues (over on GitHub)
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1: PEP 581 (Using GitHub issues for CPython) is accepted</strong></p> <ul> <li><a href="">PEP 581</a></li> <li><a href="">The email announcing the acceptance.</a></li> <li>“The migration will be a large effort, with much planning, development, and testing, and we welcome volunteers who wish to help make it a reality. I look forward to your contributions on <a href="">PEP 588</a> and the actual work of migrating issues to GitHub.” — Barry Warsaw</li> </ul> <p><strong>Michael</strong> <strong>#2</strong>: <a href=""><strong>Replace Nested Conditional with Guard Clauses</strong></a></p> <ul> <li>Deeply nested code is problematic (does it have deodorant — err comments?)</li> <li>But what can you do? Guard clauses!</li> <li>See <a href="">Martin Fowler’s article</a> and <a href="">this one.</a></li> </ul> <pre><code> # BAD! def checkout(user): shipping, express = [], [] if user is not None: for item in user.cart: if item.is_available: shipping.append(item) if item.express_selected: express.append(item) return shipping, express </code></pre> <pre><code> # BETTER! def checkout(user): shipping, express = [], [] if user is None: return shipping, express for item in user.cart: if not item.is_available: continue shipping.append(item) if item.express_selected: express.append(item) return shipping, express </code></pre> <p><strong>Brian #3:</strong> <a href=""><strong>Things you’re probably not using in Python 3 – but should</strong></a></p> <ul> <li>Vinko Kodžoman</li> <li>Some of course items: <ul> <li>f-strings</li> <li>Pathlib (side note. <a href="">pytest</a> <a href="">tmp_path</a> <a href="">fixture</a> creates temporary directories and files with PathLib)</li> <li>data classes</li> </ul></li> <li>Some I’m warming to: <ul> <li>type hinting</li> </ul></li> <li>And those I’m really glad for the reminder of:</li> <li>enumerations</li> </ul> <pre><code> from enum import Enum, auto class Monster(Enum): ZOMBIE = auto() WARRIOR = auto() BEAR = auto() print(Monster.ZOMBIE) # Monster.ZOMBIE </code></pre> <ul> <li>built in lru_cache: easy memoization with the <code>functools.lru_cache</code> decorator.</li> </ul> <pre><code> @lru_cache(maxsize=512) def fib_memoization(number: int) -&gt; int: ... </code></pre> <ul> <li>extended iterable unpacking</li> </ul> <pre><code> &gt;&gt;&gt; head, *body, tail = range(5) &gt;&gt;&gt; print(head, body, tail) 0 [1, 2, 3] 4 &gt;&gt;&gt; py, filename, *cmds = "python3.7 -n 5 -l 15".split() &gt;&gt;&gt; cmds ['-n', '5', '-l', '15'] &gt;&gt;&gt; first, _, third, *_ = range(10) &gt;&gt;&gt; first, third (0, 2) </code></pre> <p><strong>Michael #4:</strong> <a href=""><strong>The Python Arcade Library</strong></a></p> <ul> <li>Arcade is an easy-to-learn Python library for creating 2D video games. It is ideal for people learning to program, or developers that want to code a 2D game without learning a complex framework.</li> <li>Minesweeper games, hangman, <a href=""><strong>platformer games</strong></a> in general.</li> <li>Check out <a href="">Sample Games Made With The Arcade Library</a> too</li> <li>Includes physics and other goodies</li> <li>Based on OpenGL</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Teaching a kid to code with Pygame Zero</strong></a></p> <ul> <li>Matt Layman</li> <li>Scratch too far removed from coding.</li> <li>Using Mu to simplify coding interface. <ul> <li>comes with a built in Python.</li> <li>Pygame Zero preinstalled</li> </ul></li> <li>“<em>[Pygame Zero] is intended for use in education, so that teachers can teach basic programming without needing to explain the Pygame API or write an event loop.”</em></li> <li>Initial 29 line game taught: <ul> <li>naming things and variables</li> <li>mutability and fiddling with “constants” to see the effect</li> <li>functions and side effects</li> <li>state and time</li> <li>interactions and mouse events</li> </ul></li> <li>Article also includes some tips on how to behave as the adult when working with kids and coding.</li> </ul> <p><strong>Michael #6:</strong> <strong>Follow up on GIL / PEP 554</strong></p> <ul> <li><a href="">Has the Python GIL been slain?</a> by Anthony Shaw</li> <li>multithreading in CPython is easy, but it’s not truly concurrent, and multiprocessing is concurrent but has a significant overhead.</li> <li>Because Interpreter state contains the memory allocation arena, a collection of all pointers to Python objects (local and global), sub-interpreters in PEP 554 cannot access the global variables of other interpreters.</li> <li>the way to share objects between interpreters would be to serialize them and use a form of IPC (network, disk or shared memory). All options are fairly inefficient</li> <li>But: <a href="">PEP 574 proposes a new pickle</a> protocol (v5) which has support for allowing memory buffers to be handled separately from the rest of the pickle stream.</li> <li>When? Pickle v5 and shared memory for multiprocessing will likely be Python 3.8 (October 2019) and sub-interpreters will be between 3.8 and 3.9.</li> </ul> <p><strong>Extras</strong></p> <p>Brian: </p> <ul> <li><a href="">PyCon 2019 videos are available</a> <ul> <li>So grateful for this. Already watched a couple, including Ant’s awesome talk about <a href="">complexity and wily</a>.</li> </ul></li> <li>pytest and hypothesis show up in the new <a href="">Pragmatic Programmer</a> book.</li> </ul> <p>Michael:</p> <ul> <li><a href=""><strong>100 Days of Web</strong></a> course is out!</li> <li><a href=""><strong>Effective PyCharm book</strong></a></li> <li>New release of our <a href=""><strong>Android and iOS apps</strong></a>.</li> </ul> <p><strong>Jokes</strong></p> <ul> <li>MK → Waiter: <strong>Would you like coffee or tea?</strong> Programmer: <strong>Yes</strong>.</li> </ul>
May 21, 2019
#130 Python.exe now shipping with Windows 10
<p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <p>Folks this one is light on notes since we did it live. Enjoy the show!</p> <p><strong>Special guests</strong></p> <ul> <li><a href="">Emily Morehouse</a></li> <li><a href="">Steve Dower</a></li> </ul> <p><strong>Topics</strong></p> <ul> <li>Brian #1: <a href="">pgcli</a></li> <li>Michael #2: <a href="">Papermill</a></li> <li>Emily #3: <a href="">Python Language Summit</a></li> <li>Steve #4: <a href="">Python in Windows 10</a></li> </ul>
May 14, 2019
#129 Maintaining a Python Project when it’s not your job
<p>Sponsored by DigitalOcean: <a href=""></a></p> <p><strong>Brian #1: Maintaining a Python Project when it’s not your job</strong></p> <p><strong>Paul #2: Python in 1994</strong></p> <ul> <li><a href=""></a></li> </ul> <p><strong>Barry #3 Python leadership in 2019</strong></p> <p><strong>Michael #4: Textblob</strong></p> <ul> <li><a href=""></a></li> </ul>
May 06, 2019
#128 Will the GIL be obsolete with PEP 554?
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Solving Algorithmic Problems in Python with pytest</strong></a></p> <ul> <li>Adam Johnson</li> <li>How to utilize pytest to set up quick test cases for coding challenges, like <a href="">Project Euler</a> or <a href="">Advent of Code</a>.</li> <li>Moving the specification and examples in the challenge description into test cases.</li> <li>Running the tests with a stub implementation and understanding the failure output.</li> <li>Gradually building up a working solution.</li> <li>Nice demo of how little code it takes to write quick test cases.</li> <li>Also a cool idea to use challenge sites and platforms as TDD/test first practice, as well as practice converting specifications into test cases.</li> </ul> <p><strong>Michael #2: <a href="">DepHell -- project management for Python</a></strong></p> <ul> <li>via <a href=""><strong>@dreigelb</strong></a></li> <li>Why it is better than all other tools: <ol> <li><strong>Format agnostic</strong>. You can use DepHell with your favorite format:, requirements.txt, Pipfile, poetry. DepHell supports them all and much more.</li> <li><strong>Use your favorite tool on any project</strong>. Want to install a poetry based project, but don't like poetry? Just say DepHell to convert project meta information into and install it with pip. Or directly work with the project from DepHell, because DepHell can do everything what you usually want to do with packages.</li> <li><strong>DepHell doesn't try to replace your favorite tools</strong>. If you use poetry, you have to use poetry's file formats and commands. However, DepHell can be combined with any other tool or even combine all these tools together through formats converting. You can use DepHell, poetry and pip at the same time.</li> <li><strong>Easily extendable</strong>. Pipfile should be just another one supported format for pip. However, pip is really old and big project with many bad decisions, so, PyPA team can't just add new features in pip without fear to broke everything. This is how pipenv has been created, but pipenv has inherited almost all problems of pip and isn't extendable too. DepHell has strong modularity and can be easily extended by new formats and commands.</li> <li><strong>Developers friendly</strong>. We aren't going to place all our modules into <code>[_internal](</code>. Also, DepHell has <a href="">big ecosystem</a> with separated libraries to help you use some DepHell's parts without pain and big dependencies for your project.</li> <li><strong>All-in-one-solution</strong>. DepHell can manage dependencies, virtual environments, tests, CLI tools, packages, generate configs, show licenses for dependencies, make security audit, get downloads statistic from pypi, search packages and much more. None of your tools can do it all.</li> <li><strong>Smart dependency resolution</strong>. Sometimes pip and pipenv can't lock your dependencies. Try to execute <code>pipenv install oslo.utils==1.4.0</code>. Pipenv can't handle it, but DepHell can: <code>dephell deps add --from=Pipfile oslo.utils==1.4.0</code> to add new dependency and <code>dephell deps convert --from=Pipfile --to=Pipfile.lock</code> to lock it.</li> <li><strong>Asyncio based</strong>. DepHell doesn't support Python 2.7, and that allows us to use modern features to make network and filesystem requests as fast as possible.</li> <li><strong>Multiple environments</strong>. You can have as many environments for project as you want. Separate sphinx dependencies from your main and dev environment. Other tools like pipenv and poetry don't support it.</li> </ol></li> </ul> <p><strong>Brian #3 <a href="">Python rant: from foo import is bad</a></strong></p> <ul> <li>Mike Croucher</li> <li>I’m glad to see this post because I’m still seeing this practice a lot, even in tutorial blog posts!</li> <li>This is meaningless: <code>result = sqrt(-1)</code></li> <li>Is it: <code>math.sqrt(-1)</code>? or <code>numpy.sqrt(-1)</code> or <code>cmath.sqrt(-1)</code>? <ul> <li>or <code>scipy</code>? or <code>sympy</code>?</li> </ul></li> <li>Recommendation: <ul> <li>Never do <code>from x import *</code></li> <li>Use <code>import math</code> <ul> <li>or <code>import numpy as np</code></li> <li>or even <code>from scipy import sqrt</code> </li> </ul></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Dask</strong></a></p> <ul> <li>Dask natively scales Python </li> <li>Have numpy, pandas, and scikit-learn code that needs to go faster?</li> <li>Run these on smart clusters of servers</li> <li>Or just on your laptop</li> <li>Process more data than will fit into RAM</li> <li>Supported by… interesting to see proper support there.</li> <li>Matthew Rocklin was on <a href=""><strong>Talk Python 207 to discuss</strong></a></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Animations with Matplotlib</strong></a></p> <ul> <li>Parul Pandey</li> <li>The raindrop simulation is mesmerizing. </li> <li>Tutorial on using <code>FuncAnimation</code> to animate a sine wave <ul> <li>although, I’m not sure what the x axis means during an animation</li> </ul></li> <li>Also: <ul> <li>live updates based on changing data</li> <li>animate turning a 3D plot</li> <li>using <code>celluloid</code> package to animate <ul> <li>simple example</li> <li>animating subplots</li> <li>changing legend during animation</li> </ul></li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>PEP 554 -- Multiple Interpreters in the Stdlib</strong></a></p> <ul> <li>This proposal introduces the stdlib interpreters module. The module will be <a href="">provisional</a>. It exposes the basic functionality of subinterpreters already provided by the C-API, along with new (basic) functionality for sharing data between interpreters.</li> <li>Sharing data centers around "channels", which are similar to queues and pipes.</li> <li>Examples and use-cases: <ul> <li>Running isolated code</li> <li>In process, true parallelism </li> <li>Versioning of modules (?)</li> <li>Plugin systems</li> </ul></li> </ul> <p><strong>Extras</strong></p> <p>Michael:</p> <ul> <li>iOS Talk Python Training app is out: <a href=""><strong></strong></a></li> <li>Find us at PyCon!</li> <li><a href=""><strong>Blessings terminal API</strong></a> (from Erik Rose, via Prayson Daniel)</li> </ul> <p><strong>Jokes</strong></p> <p>via Topher Chung</p> <ul> <li>Knock knock.</li> <li>Race condition.</li> <li>Who's there?</li> </ul>
May 02, 2019
#127 That Python code is on fire!
<p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <p>Special guest: <a href="">Kenneth Reitz</a></p> <p><strong>Brian #1:</strong> <a href=""><strong>inline_python (for rust)</strong></a></p> <ul> <li>“I just made a Frankenstein's monster: Python code embedded directly in rustlang code. Should I kill it before it escapes the lab?” - <a href="">Mara Bos</a></li> <li>Writing some rust, and need a little Python?</li> <li>Maybe want to pop open a matplotlib window?</li> <li>This may be just the thing you need.</li> <li>see also: <ul> <li></li> </ul></li> </ul> <p><strong>Kenneth #2: Requests3: Under Way!</strong></p> <ul> <li>Requests 2.x that you know and love is going into CVE-only mode (which it has been for a long time).</li> <li>Requests III is a new project which will bring async/await keywords to Requests.</li> <li>installable as <code>requests3</code>.</li> <li>Type-Annotations</li> <li>Python 3.6+</li> </ul> <p><strong>Michael #3: 🔥 <a href="">Pyflame</strong></a>: <em>A Ptracing Profiler For Python</em></p> <ul> <li>Pyflame is a high performance profiling tool that generates <a href="">flame graphs</a> for Python.</li> <li>Pyflame is implemented in C++, and uses the Linux <a href="">ptrace(2)</a> system call to collect profiling information.</li> <li>It can take snapshots of the Python call stack without explicit instrumentation</li> <li>Capable of profiling embedded Python interpreters like <a href="">uWSGI</a>. </li> <li>Fully supports profiling multi-threaded Python programs.</li> <li>Why use it? <ul> <li>Pyflame usually introduces significantly less overhead than the builtin <code>profile</code> (or <code>cProfile</code>) modules, and emits richer profiling data. </li> <li>The profiling overhead is low enough that you can use it to profile live processes in production.</li> </ul></li> </ul> <p><strong>Brian #4: <a href="">flit + src</strong></a></p> <ul> <li>Currently a WIP PR.</li> <li><code>flit</code> is easy. <ul> <li>Given a module or a source package.</li> <li><code>flit init</code> creates <code>pyproject.toml</code> and <code>LICENSE</code> files.</li> <li>commit those to git</li> <li><code>flit build</code> creates a wheel</li> <li><code>flit publish</code> (builds and) publishes to whatever you have in your <code>[.pypirc](</code></li> </ul></li> <li>Changes in this PR <ul> <li>The flit project already has 2 types of projects. <ul> <li>just a module, like</li> <li>a package (directory with <code></code>), like <code>foo/</code></li> </ul></li> <li>This would add a 3rd and 4th. <ul> <li>just a module, but in src, like <code>src/</code></li> <li>a package in src, like <code>src/foo/</code></li> </ul></li> </ul></li> <li>May be cracking open a can of worms, but I’m ok with that.</li> </ul> <p><strong>Kenneth #5</strong>: $ <strong>pipx install pipenv</strong></p> <p><strong>Michael #6:</strong> <a href=""><strong></strong></a></p> <ul> <li>via Jon Bultmeyer</li> <li>Nothing to install, but works on the CLI <ul> <li>$ <a href="http://http list"><strong>http</strong></a></li> <li>$ <a href="http://http to database"><strong>http</strong></a></li> </ul></li> <li>Has a CLI client too with a proper shell</li> <li>Get started with <a href="http://http"><strong>http</strong></a></li> <li>Has a funky stealth mode too</li> <li>Editor integration VS Code &amp; Vim</li> <li><em></em> uses selected community driven cheat sheet repositories and information sources, maintained by thousands of users, developers and authors all over the world</li> </ul> <p><strong>Extras</strong></p> <p>Brian:</p> <ul> <li><a href="">vi is good for beginners</a> - fun read, for all you haters out there. <ul> <li>But use vim, not vi.</li> <li>Better yet, IdeaVim for PyCharm or VSCodeVim for VS Code.</li> </ul></li> <li><a href="">nbstripout</a> - command line tool to strip output from Jupyter Notebook files.</li> <li>We covered <a href="">pyodide</a> on <a href="">episode 93</a>, but here’s a cool article on it <ul> <li><a href="">Pyodide: Bringing the scientific Python stack to the browser</a></li> </ul></li> </ul> <p>Michael:</p> <ul> <li>PyCon AU CFP</li> <li>LIGO Blackhole collision follow up: <a href=""></a> <ul> <li>via Dave Kirby and Matthew Feickert</li> </ul></li> <li><a href=""></a> like Bullet but for windows too <ul> <li>via Sander Teunissen</li> </ul></li> </ul> <p>Kenneth (optional):</p> <ul> <li>PyColorado CFP</li> <li>PyOhio CFP</li> <li>PyRemote!</li> </ul> <p><strong>Jokes</strong></p> <p>Don’t know if I’ll do all of these, but I like them. 🙂 Brian and Kenneth, feel free to add yours if you have some!</p> <p>MK: Ubuntu users are apt to get these jokes. MK: How many programmers does it take to kill a cockroach? Two: one holds, the other installs Windows on it. MK: A programmer had a problem. He thought to himself, 'I know, I'll solve it with threads!'. has Now problems. two he</p> <p>(mildly offensive) KR: What’s the difference between a musician and a pizza? A pizza can feed a family of four.</p> <p>(In collaboration with Jonatan Skogsfors) Python used to be directed by the BDFL, Guido. Now it’s directed by a steering council, GUIDs[0:4].</p>
Apr 25, 2019
#126 WebAssembly comes to Python
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>Special guest: <strong><a href="">Cecil Philip</a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>Python Used to Take Photo of Black Hole</strong></a></p> <ul> <li>Lots of people talking about this. The link I’m including is a quick write up by Mike Driscoll.</li> <li>From now on these conversations can happen: <ul> <li>“So, what can you do with Python?”</li> <li>“Well, it was used to help produce the worlds first image of a black hole. Your particular problem probably isn’t as complicated as that, so Python should work fine.”</li> </ul></li> <li>Projects listed in the paper: “<a href="">First M87 Event Horizon Telescope Results. III. Data Processing and Calibration</a>”: <ul> <li><a href="">Numpy</a> (van der Walt et al. 2011)</li> <li><a href="">Scipy</a> (Jones et al. 2001)</li> <li><a href="">Pandas</a> (McKinney 2010)</li> <li><a href="">Jupyter</a> (Kluyver et al. 2016)</li> <li><a href="">Matplotlib</a> (Hunter 2007).</li> <li><a href="">Astropy</a> (The Astropy Collaboration et al. 2013, 2018)</li> </ul></li> </ul> <p><strong>Cecil #2:</strong> <a href=""><strong>Wasmer - Python Library for executing WebAssembly binaries</strong></a></p> <ul> <li><a href="">WebAssembly</a> (Wasm) enables high level languages to target a portable format that runs in the web</li> <li>Tons of languages compile down to Wasm but Wasmer enables the consumption of Wasm in python</li> <li>This enables an interesting use case for using Wasm as a way to leverage code between languages</li> </ul> <p><strong>Michael #3:</strong> <a href=""><strong>Cooked Input</strong></a></p> <ul> <li>cooked_input is a Python package for getting, cleaning, converting, and validating command line input.</li> <li>Name comes from <code>input</code> / <code>raw_input</code> (unvalidated) and cooked input (validated)</li> <li>Beginner’s can use the provided convenience classes to get simple inputs from the user.</li> <li>More complicated command line application (CLI) input can take advantage of <code>cooked_input</code>’s ability to create commands, menus and data tables.</li> <li>All sorts of cool validates and cleaners</li> <li>Examples</li> </ul> <pre><code> cap_cleaner = ci.CapitalizationCleaner(style=ci.ALL_WORDS_CAP_STYLE) ci.get_string(prompt="What is your name?", cleaners=[cap_cleaner]) </code></pre> <pre><code> &gt;&gt;&gt; ci.get_int(prompt="How old are you?", minimum=1) How old are you?: abc "abc" cannot be converted to an integer number How old are you?: 0 "0" too low (min_val=1) How old are you?: 67 67 </code></pre> <p><strong>Brian #4:</strong> <a href=""><strong>JetBrains and PyCharm officially collaborating with Anaconda</strong></a></p> <ul> <li>PyCharm 2019.1.1 has some improvements for using Conda environments. <ul> <li>Fixed various bugs related to creating Conda envs and installing packages into them.</li> </ul></li> <li>Special distribution of PyCharm: <a href="">PyCharm for Anaconda</a> with enhanced Anaconda support.</li> <li>I’m using PyCharm Pro with vim emulation this week to edit a notebook based presentation. I might run them in Jupyter, or just run it in PyCharm, but editing with all my normal keyboard shortcuts is awesome.</li> </ul> <p><strong>Cecil #5:</strong> <a href="">Building a Serverless IoT Solution with Python Azure Functions and SignalR</a></p> <ul> <li>Interesting blog post on using serverless, IoT, real-time messaging to create a live dashboard</li> <li>Shows how to create a serverless function in Python to process IoT data</li> <li>There’s tons of DIY applications for using this technique at home </li> <li>The Dashboard is a static website using D3 for charting.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>multiprocessing.shared_memory — Provides shared memory for direct access across processes</strong></a></p> <ul> <li>New in Python 3.8</li> <li>This module provides a class, <a href="">SharedMemory</a>, for the allocation and management of shared memory to be accessed by one or more processes on a multicore or symmetric multiprocessor (SMP) machine.</li> <li>The <code>ShareableList</code> looks nice to use.</li> </ul> <p><strong>Extras</strong></p> <p>Brian:</p> <ul> <li>Getting ready for PyCon with STICKERS. Yeah, baby. Come see us at PyCon. I’ll also be bringing some copies of <a href="">Python Testing with pytest</a>, if anyone doesn’t already have a copy.</li> <li>Lots of interviews going on for <a href="">Test &amp; Code</a>, and some will happen at PyCon.</li> </ul> <p>Cecil: </p> <ul> <li><a href="">Attendee Detector Workshop</a> </li> <li><a href="">Talk Python training app on Android</a></li> </ul> <p>Michael:</p> <ul> <li>Guido van Rossum <a href=""><strong>interviewed on MIT’s AI podcast</strong></a> via Tony Cappellini </li> <li><a href=""><strong>Visual Studio IntelliCode for VS &amp; VS Code</strong></a></li> <li><a href=";"><strong>Showing a Craigslist scammer who's boss using Python</strong></a> via Dan Koster</li> </ul> <p><strong>Jokes</strong></p> <p>Brian: To understand recursion you must first understand recursion.</p> <p>Michael: A programmer was found dead in the shower. Next to their body was a bottle of shampoo with the instructions 'Lather, Rinse and <strong>Repeat</strong>'.</p>
Apr 19, 2019
#125 Will you conquer the deadlock empire?
<p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>My How and Why: pyproject.toml &amp; the 'src' Project Structure</strong></a></p> <ul> <li>Brian Skinn</li> <li>pyproject.toml <ul> <li>but with setuptools, instead of flit or poetry</li> <li>with a <code>src</code> dir</li> <li>and tox and black</li> </ul></li> <li>all the bits and pieces to make all of this work</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>The Deadlock Empire: Slay dragons, master concurrency!</strong></a></p> <ul> <li>A game to test your thread safety and skill!</li> <li>Deadlocks occur in code when two threads end up trying to enter two or more locks (RLocks please!)</li> <li>Consider <code>lock_a</code> and <code>lock_b</code></li> <li>Thread one enters <code>lock_a</code> and will soon enter <code>lock_b</code></li> <li>Thread two enters <code>lock_b</code> and will soon enter <code>lock</code>_a</li> <li>Imagine transferring money between two accounts, each with a lock, and each thread does this in opposite order.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Cog 3.0</strong></a></p> <ul> <li>Ned Batchelder’s <code>cog</code> gets an update (last one was a few years ago).</li> <li>“<a href="">Cog</a> … finds snippets of Python in text files, executes them, and inserts the result back into the text. It’s good for adding a little bit of computational support into an otherwise static file.”</li> <li>Development moved from Bitbucket to GitHub. </li> <li>Travis and Appveyor CI.</li> <li>The biggest functional change is that errors during execution now get reasonable tracebacks that don’t require you to reverse-engineer how cog ran your code.</li> <li><a href="">mutmut</a> mutation testing added. Cool.</li> <li>What I want to know more about is this statement: “…now I use it for making all my presentations”. Very cool idea.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>StackOverflow 2019 Developer Survey Results</strong></a></p> <ul> <li>More good news for Python</li> <li>Lots of focus on gender in this one</li> <li><a href="">Contributing to Open Source</a> <ul> <li>About 65% of professional developers on Stack Overflow contribute to open source projects once a year or more. Involvement in open source varies with language. Developers who work with Rust, WebAssembly, and Elixir contribute to open source at the highest rates, while developers who work with VBA, C#, and SQL do so at about half those rates.</li> </ul></li> <li><a href="">Competence and Experience</a> <ul> <li>We see evidence here among the most junior developers for <em>impostor syndrome</em>, pervasive patterns of self-doubt, insecurity, and fear of being exposed as a fraud. Among our respondents, men grew more confident much more quickly than gender minorities.</li> </ul></li> <li><a href="">Programming, Scripting, and Markup Languages</a> <ul> <li>Python edges out Java, second only to JavaScript (and two non-programming languages)</li> </ul></li> <li><a href="">Databases</a> <ul> <li>MySQL, Postgres, Microsoft SQL Server, SQLite, MongoDB</li> </ul></li> <li><a href="">Most Loved, Dreaded, and Wanted Languages</a> <ul> <li>Loved: Rust, Python</li> <li>Wanted: Python, JavaScript</li> <li>Dreaded: VBA, ObjectiveC</li> </ul></li> <li><a href="">Most Loved, Dreaded, and Wanted Databases</a> <ul> <li>Loved: Postgres</li> <li>Wanted: MongoDB</li> </ul></li> <li><a href="">Most Popular Development Environments</a> <ul> <li>VS Code is crushing it</li> </ul></li> <li><a href="">How Technologies Are Connected</a> is just interesting</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Cuv’ner</strong></a> <strong>“</strong>A commanding view of your test-coverage"</p> <ul> <li>Coverage visualizations on the console.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Mobile apps launched</strong></a></p> <ul> <li>The tech (sadly only 50% Python) <ul> <li>Xamarin, Mono, and C# on the device-side</li> <li>Python, Pyramid, and MongoDB on the server-side</li> </ul></li> <li>90% code sharing or higher</li> <li>Native applications</li> <li>Build the prototype myself on Windows</li> <li>Hired Giorgi via <a href=""><strong>TopTal</strong></a> <ul> <li>Get your own developer or get some freelancing work and support my app progress with my referral code: <a href=""><strong></strong></a> </li> </ul></li> <li>Dear mobile app developers: You have my sympathy!</li> <li>Try the app at <a href=""><strong></strong></a> Comes with 2 free courses for anyone who logs in.</li> <li>Android only at the moment but not for long</li> </ul> <p><strong>Extras</strong></p> <p><strong>Brian</strong>:</p> <ul> <li>Python Bytes Patreon page is up: <strong><a href=""></a></strong></li> </ul> <p><strong>Michael</strong>:</p> <ul> <li>PyCon Booth</li> <li><a href=""><strong>XKCD Plots in Matplotlib</strong></a> with <a href=""><strong>examples</strong></a> via Tim Harrison</li> <li><a href=""><strong>Fira Code Retina and Font Ligatures</strong></a></li> <li>The EuroSciPy 2019 Conference will take place from September 2 to September 6 in Bilbao, Spain</li> </ul> <p><strong>Jokes</strong></p> <ul> <li>“When your hammer is C++, everything begins to look like a thumb.”</li> <li>“Why don't jokes work in octal? Because 7 10 11” <ul> <li>Over explained: Why is 6 afraid of 7. Cuz 7 8 9.</li> <li>Follow on: Why did 7 eat 9? He was trying to eat 3^2 meals.</li> </ul></li> <li>I've been using Vim for a long time now, mainly because I can't figure out how to exit.</li> </ul>
Apr 13, 2019
#124 This is not the None you're looking for
<p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <pstrong>Brian #1:</strong> <a href=""><strong>pytest 4.4.0</strong></a> </p> <ul> <li>Lots of amazing new features here (at least for testing nerds)</li> <li><a href="">testpaths</a> displayed in output, if used. <ul> <li><code>pytest.ini</code> setting that allows you to specify a list of directories or tests (relative to test rootdir) to test. (can speed up test collection).</li> </ul></li> <li>Lots of goodies for plugin writers.</li> <li>Internal changes to allow subtests to work with a new plugin, <a href="">pytest-subtests</a>.</li> <li>Just started playing with it, but I’m excited already. Planning on a full <a href="">Test &amp; Code</a> episode after I play with it a bit more.</li> </ul> <pre><code> # unittest example: class T(unittest.TestCase): def test_foo(self): for i in range(5): with self.subTest("custom message", i=i): self.assertEqual(i % 2, 0) # pytest example: def test(subtests): for i in range(5): with subtests.test(msg="custom message", i=i): assert i % 2 == 0 </code></pre> <pstrong>Michael #2:</strong> <a href=""><strong>requests-async</strong></a></p> <ul> <li>async-await support for requests</li> <li>Just finished talking with Kenneth Reitz, native async coming to requests, but awhile off</li> <li>Nice interm solution</li> <li>Requires modern Python (3.6)</li> <li>Interesting Flask, Quart, Starlette, etc. framework wrapper for testing</li> </ul> <pstrong>Brian #3:</strong> <strong>Reasons why PyPI should not be a service</strong></p> <ul> <li>Dustin Ingram’s article: <a href=""><strong>PyPI as a Service</strong></a></li> <li>“Layoffs at JavaScript package registry raise questions about fate of community resource” - <a href="">The Register article</a></li> <li>Apparently PyPI gets requests for a private form of their service regularly, but there are problems with that.</li> <li>Currently a non-profit project under the PSF. That may be hard to maintain if they have a for-profit part.</li> <li>Donated services and infrastructure of more than $1M/year would be hard to replace.</li> <li>There are already other package repository options. Although there is probably room for others to compete.</li> <li>Currently run by volunteers for the most part. (&lt;1 employee). Don’t think they would stick around to volunteer for a for-profit enterprise.</li> <li>conclusion: not impossible, but probably not worth it.</li> </ul> <pstrong>Michael #4: <a href="">Jupyter in the cloud</strong></a></p> <ul> <li>Six easy ways to run your Jupyter Notebook in the cloud by Kevin Markham</li> <li>six services you can use to easily run your Jupyter notebook in the cloud. All of them have the following characteristics: <ul> <li>They don't require you to install anything on your local machine.</li> <li>They are completely free (or they have a free plan).</li> <li>They give you access to the Jupyter Notebook environment (or a Jupyter-like environment).</li> <li>They allow you to import and export notebooks using the standard .ipynb file format.</li> <li>They support the Python language (and most support other languages as well).</li> </ul></li> <li><a href="">Binder</a> is a service provided by the Binder Project, which is a member of the Project Jupyter open source ecosystem. It allows you to input the URL of any public Git repository, and it will open that repository within the native Jupyter Notebook interface.</li> <li><a href="">Kaggle</a> is best known as a platform for data science competitions. However, they also provide a free service called <a href="">Kernels</a> that can be used independently of their competitions.</li> <li><a href="">Google Colaboratory</a>, usually referred to as "Google Colab," is available to anyone with a Google account. As long as you are signed into Google, you can quickly get started by creating an empty notebook, uploading an existing notebook, or importing a notebook from any public GitHub repository.</li> <li>To get started with <a href="">Azure Notebooks</a>, you first sign in with a Microsoft or Outlook account (or create one). The next step is to create a "project", which is structured identically to a GitHub repository: it can contain one or more notebooks, Markdown files, datasets, and any other file you want to create or upload, and all of these can be organized into folders.</li> <li><a href="">CoCalc</a>, short for "collaborative calculation", is an online workspace for computation in Python, R, Julia, and many other languages. It allows you to create and edit Jupyter Notebooks, Sage worksheets, and LaTeX documents.</li> <li><a href="">Datalore</a> was created by JetBrains, the same company who makes PyCharm (a popular Python IDE). Getting started is as easy as creating an account, or logging in with a Google or JetBrains account. You can either create a new Datalore "workbook" or upload an existing Jupyter Notebook.</li> </ul> <pstrong>Brian #5:</strong> <strong>Jupyter Notebook tutorials</strong></p> <ul> <li>These are from Dataquest</li> <li><a href=""><strong>Jupyter Notebook for Beginners: A Tutorial</strong></a> <ul> <li>Incredibly gentle, concise, useful tutorial to get started quickly.</li> <li>Installation, creating, and running with server and browser.</li> <li>Discussion of .ipynb files</li> <li>Overview of interface, cells, shortcuts, markdown.</li> <li>Kernels</li> <li>Starting with data. Importing appropriate libraries, loading data.</li> <li>Save and checkpoint</li> <li>looking at data, graphing/plotting data</li> <li>Sharing notebooks: exporting, using github and gists, nbviewer, </li> </ul></li> <li><a href=""><strong>Tutorial: Advanced Jupyter Notebooks</strong></a> <ul> <li>shell commands</li> <li>basic magics</li> <li>autosaving</li> <li>matplotlib inline</li> <li>debugging in Jupyter </li> <li>(Brian: Gak! Maybe switch to PyCharm for debugging)</li> <li>using timeit</li> <li>rendering theml, latex, other languages in cells.</li> <li>logging, extensions</li> <li>charts with seaborn</li> <li>macros</li> <li>loading, importing and running external code and snippets.</li> <li>scripted execution, even on the command line</li> <li>parametrization with env variables</li> <li>styling, hiding cells, working with databases</li> </ul></li> </ul> <pstrong>Michael #6: <a href="">Unique sentinel values, identity checks, and when to use object() instead of None</strong></a></p> <ul> <li>By Trey Hunner</li> <li>In Python (and in programming in general), you’ll need an object which can be uniquely identified. Sometimes this unique object represents a <strong>stop value</strong> or a <strong>skip value</strong> and sometimes it’s an <strong>initial value</strong>.</li> <li>Often this is None, but there are plenty of gotchas packed in there.</li> <li>Nice example of re-implementing min.</li> <li>Make sure to leverage <code>is</code> rather than <code>==</code></li> </ul> <pre><code> initial = object() # ... if minimum is not initial: return minimum # ... </code></pre> <pstrong>Extras</strong></p> <p><strong>Brian</strong></p> <ul> <li><a href="">pytest-neo</a></li> </ul> <p><strong>Michael</strong></p> <ul> <li><a href="">Responder course</a></li> <li><a href="">AceJump</a> for IntelliJ platforms (including PyCharm)</li> </ul> <pstrong>Jokes</strong></p>
Apr 05, 2019
#123 Time to right the py-wrongs
<p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>Deconstructing</strong></a></p> <ul> <li>Brett Cannon</li> <li>Breakdown of the infamous <a href="">xkcd comic poking fun at the authors Python Environment on his computer</a>. <ul> <li>The interpreters listed</li> <li>Homebrew description</li> <li> binaries</li> <li>A discussion of pip, easy_install</li> <li>The paths and the <code>$PATH</code> and <code>$PYTHONPATH</code></li> </ul></li> <li>Actually quite an educational history lesson, and the abuse some people put their computers through.</li> <li>“So the next time someone decides to link to this comic as proof that Python has a problem, you can say that it's actually Randall's problem.”</li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Python package as a CLI option</strong></a></p> <ul> <li>Wanted to make <a href=""><strong>this little app</strong></a> available via a CLI as a dedicated command. Really tired of <code>python3</code> or <code>./</code></li> <li>Turns out, pip and Python already solve this problem, if you structure your package correctly</li> <li>Thanks to everyone on Twitter!</li> <li>The trick turns out to be to have entrypoints in your package</li> </ul> <pre><code> entry_points = { "console_scripts": ['bootstrap = bootstrap.bootstrap:main'] } ... </code></pre> <p>This should even register it with <code>pipx install package</code> ;)</p> <pstrong>Brian #3</strong>: <a href=""><strong>pyright</strong></a></p> <ul> <li>a Microsoft static type checker for the Python language.</li> <li>“Pyright was created to address gaps in existing Python type checkers like <a href="">mypy</a>.”</li> <li>5x faster than mypy</li> <li>meant for large code bases</li> <li>written in TypeScript and runs within node.</li> </ul> <pstrong>Michael #4:</strong> <a href=""><strong>Refactoring Python Applications for Simplicity</strong></a></p> <ul> <li>If you can write and maintain clean, simple Python code, then it’ll save you lots of time in the long term. You can spend less time testing, finding bugs, and making changes when your code is well laid out and simple to follow.</li> <li><strong>Is your code complex</strong>?</li> <li>Metrics for Measuring Complexity <ul> <li><strong>Lines of Code</strong></li> <li><strong>Cyclomatic complexity</strong> is the measure of how many independent code paths there are through your application.</li> <li><strong>Maintainability Index</strong></li> </ul></li> <li><strong>Refactoring</strong>: The technique of changing an application (either the code or the architecture) so that it behaves the same way on the outside, but internally has improved.</li> <li>Nice overview of tooling (PyCharm, VS Code plugins, etc)</li> <li>Anti-patterns and ways out of them (best part of the article IMO)</li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>FastAPI</strong></a></p> <ul> <li>Thanks Colin Sullivan for suggesting the topic</li> <li>“<em>FastAPI framework, high performance, easy to learn, fast to code, ready for production”</em></li> <li>“Sales pitch / key features: <ul> <li><strong>Fast</strong>: Very high performance, on par with <strong>NodeJS</strong> and <strong>Go</strong> (thanks to Starlette and Pydantic). <a href="">One of the fastest Python frameworks available</a>.</li> <li><strong>Fast to code</strong>: Increase the speed to develop features by about 200% to 300%. (estimated)</li> <li><strong>Fewer bugs</strong>: Reduce about 40% of human (developer) induced errors. (estimated)</li> <li><strong>Intuitive</strong>: Great editor support. Completion everywhere. Less time debugging.</li> <li><strong>Easy</strong>: Designed to be easy to use and learn. Less time reading docs.</li> <li><strong>Short</strong>: Minimize code duplication. Multiple features from each parameter declaration. Fewer bugs.</li> <li><strong>Robust</strong>: Get production-ready code. With automatic interactive documentation.</li> <li><strong>Standards-based</strong>: Based on (and fully compatible with) the open standards for APIs: <a href="">OpenAPI</a>(previously known as Swagger) and <a href="">JSON Schema</a>.”</li> </ul></li> <li>uses: <ul> <li><a href="">Starlette</a> for the web parts.</li> <li><a href="">Pydantic</a> for the data parts.</li> </ul></li> <li>document REST apis with both <ul> <li>Swagger</li> <li>ReDoc</li> </ul></li> <li>looks like quite a fun contender in the “put together a REST API quickly” set of solutions out there.</li> <li>Just the front page demo is quite informative. There’s also a tutorial that seems like it might be a crash course in API best practices.</li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>Bleach: stepping down as maintainer</strong></a></p> <ul> <li>by Will Kahn-Greene</li> <li><a href="">Bleach</a> is a Python library for sanitizing and linkifying text from untrusted sources for safe usage in HTML.</li> <li>A retrospective on OSS project maintenance</li> <li>Picked up maintenance of the project because <ul> <li>I was familiar with it</li> <li>current maintainer really wanted to step down</li> <li>Mozilla was using it on a bunch of sites</li> <li>I felt an obligation to make sure it didn't drop on the floor and I knew I could do it.</li> </ul></li> <li>Never really <em>liked</em> working on Bleach</li> <li>He did a bunch of work on a project I don't really use, but felt obligated to make sure it didn't fall on the floor, that has a pain-in-the-ass problem domain. Did that for 3+ years.</li> <li>Is [he] getting paid to work on it? Not really.</li> <li>Does [he] like working on it? No.</li> <li>Seems like [he] shouldn't be working on it anymore.</li> </ul> <pstrong>Extras</strong></p> <p><strong>Brian</strong></p> <ul> <li><strong><a href="">sleepsort</a></strong></li> </ul> <p><strong>Michael</strong>: </p> <ul> <li><a href=""><strong>Passbolt</strong></a></li> <li><a href=""><strong>Python 3.7.3 is now available</strong></a></li> <li><a href=""><strong>stackroboflow</strong></a> via Alexander Allori</li> </ul> <pstrong>Joke</strong></p>
Mar 29, 2019
#122 Give Me Back My Monolith
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <strong>Combining and separating dictionaries</strong></p> <ul> <li><a href="">PEP 584 -- Add + and - operators to the built-in dict class.</a> <ul> <li>Steven D'Aprano</li> <li>Draft status, just created 1-March-2019</li> <li>d1 + d2 would merge d2 into d1 <ul> <li>like <code>{**d1, **d2}</code></li> <li>or on two lines</li> </ul></li> </ul></li> </ul> <pre><code> d = d1.copy() d.update(d2) </code></pre> <ul> <li>of note, (d1 + d2) != (d2 + d1)</li> <li>Currently no subtraction equivalent <ul> <li><a href="">Guido’s preference of + over |</a></li> <li>Related, <a href="">Why operators are useful</a> - also by Guido</li> </ul></li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Why I Avoid Slack</strong></a></p> <ul> <li>by Matthew Rocklin</li> <li>I avoid interacting on Slack, especially for technical conversations around open source software. </li> <li>Instead, I encourage colleagues to have technical and design conversations on GitHub, or some other system that is public, permanent, searchable, and cross-referenceable.</li> <li>Slack is fun but, internal real-time chat systems are, I think, bad for productivity generally, especially for public open source software maintenance.</li> <li>Prefer GitHub because I want to <ul> <li><strong>Engage collaborators</strong> that aren’t on our Slack</li> <li><strong>Record the conversation</strong> in case participants change in the future.</li> <li><strong>Serve the silent majority</strong> of users who search the web for answers to their questions or bugs.</li> <li><strong>Encourage thoughtful discourse</strong>. Because GitHub is a permanent record it forces people to think more before they write.</li> <li><strong>Cross reference issues</strong>. Slack is siloed. It doesn’t allow people to cross reference people or conversations across Slacks</li> </ul></li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>Hunting for Memory Leaks in Python applications</strong></a></p> <ul> <li>Wai Chee Yau</li> <li>Conquering memory leaks and spikes in Python ML products at Zendesk.</li> <li>A quick tutorial of some useful memory tools</li> <li>The <code>memory_profiler</code> package and <code>matplotlib</code> to visualize memory spikes.</li> <li>Using <code>muppy</code> to heap dump at certain places in the code.</li> <li><code>objgraph</code> to help memory profiling with object lineage.</li> <li>Some tips when memory leak/spike hunting: <ul> <li>strive for quick feedback</li> <li>run memory intensive tasks in separate processes</li> <li>debugger can add references to objects</li> <li>watch out for packages that can be leaky <ul> <li><code>pandas</code>? really?</li> </ul></li> </ul></li> </ul> <pstrong>Michael #4:</strong> <a href=""><strong>Give Me Back My Monolith</strong></a></p> <ul> <li>by <a href="">Craig Kerstiens</a></li> <li>Feels like we’re starting to pass the peak of the hype cycle of microservices</li> <li>We’ve actually seen some migrations from <a href="">micro-services back to a monolith</a>.</li> <li>Here is a rundown of all the things that were simple that you now get to re-visit</li> <li>Setup went from intro chem to quantum mechanics <ul> <li>Onboarding a new engineering, at least for an initial environment would be done in the first day. As we ventured into micro-services onboarding time skyrocketed</li> </ul></li> <li>So long for understanding our systems <ul> <li>Back when we had monolithic apps if you had an error you had a clear stacktrace to see where it originated from and could jump right in and debug. Now we have a service that talks to another service, that queues something on a message bus, that another service processes, and then we have an error.</li> </ul></li> <li>If we can’t debug them, maybe we can test them</li> <li>All the trade-offs are for a good reason. Right?</li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>Famous Laws Of Software Development</strong></a></p> <ul> <li>Tim Sommer</li> <li>13 “laws” of software development, including <ul> <li>Hofstadter’s Law: “It always takes longer than you expect, even when you take into account Hofstadter's Law.”</li> <li>Conway’s Law: “Any piece of software reflects the organizational structure that produced it.”</li> <li>The Peter Principle: “In a hierarchy, every employee tends to rise to his level of incompetence.”</li> <li>Ninety-ninety rule: “The first 90% of the code takes 10% of the time. The remaining 10% takes the other 90% of the time”</li> </ul></li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>Beer Garden Plugins</strong></a></p> <ul> <li>A powerful plugin framework for converting your functions into composable, discoverable, production-ready services with minimal overhead.</li> <li>Beer Garden makes it easy to turn your functions into REST interfaces that are ready for production use, in a way that’s accessible to anyone that can write a function.</li> <li>Based on MongoDB, Rabbit MQ, &amp; modern Python</li> <li>Nice docker-compose option too</li> </ul> <pstrong>Extras</strong></p> <p>Michael:</p> <ul> <li><a href="">Firefox Send</a></li> <li>Ethical ads on Python Bytes (and Talk Python)</li> </ul> <p>Brian: </p> <ul> <li><a href="">T&amp;C 69: The Pragmatic Programmer — Andy Hunt</a> <ul> <li>not up yet, but will be before this episode is released</li> </ul></li> </ul> <pstrong>Jokes</strong></p> <ul> <li><p>From <a href="">Derrick Chambers</a></p> <p>“What do you call it when a python programmer refuses to implement custom objects? self deprivation! Sorry, that joke was really classless.”</p></li> <li><p>via <a href="">pyjokes</a>: I had a problem so I thought I'd use Java. Now I have a <code>ProblemFactory</code>.</p></li> </ul>
Mar 22, 2019
#121 python2 becomes self-aware, enters fifth stage of grief
<p>Sponsored by Datadog: <a href=""><em></em></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>Futurize</strong></a> <strong>and</strong> <a href=""><strong>Auto-Futurize</strong></a></p> <ul> <li>Staged automatic conversion from Python2 to Python3 with <a href="">futurize</a> from <ul> <li><code>pip install future</code></li> </ul></li> <li>Stages: <ul> <li>1: safe fixes: <ul> <li>exception syntax, print function, object base class, iterator syntax, key checking in dictionaries, and more</li> </ul></li> <li>2: Python 3 style code with wrappers for Python 2 <ul> <li>more risky items to change</li> <li>separating text from bytes, quite a few more</li> </ul></li> <li>very modular and you can be more aggressive and more conservative with flags.</li> </ul></li> <li>Do that, but between each step, run tests, and only continue if they pass, with <a href="">auto-futurize</a> from Timothy Hopper. <ul> <li>a shell script that uses <code>git</code> to save staged changes and <code>tox</code> to test the code.</li> </ul></li> </ul> <pstrong>Michael #2:</strong> <a href=";"><strong>Tech blog writing live stream</strong></a></p> <ul> <li>via <a href="">Anthony Shaw</a></li> <li>Live stream on "technical blog writing"</li> <li>Talking about how I put articles together, research, timing and other things about layouts and narratives.</li> <li>Covers “Modifying the Python language in 6 minutes”, deep article</li> <li>Listicals, “5 Easy Coding Projects to Do with Kids”</li> <li>A little insight into what is popular.</li> <li>Question article: Why is Python Slow?</li> <li>Tourists guide to the CPython source code</li> </ul> <pstrong>Brian #3</strong>: <a href=""><strong>Try out walrus operator in Python 3.8</strong></a></p> <ul> <li>Alexander Hultnér</li> <li>The walrus operator is the assignment expression that is coming in thanks to PEP 572.</li> </ul> <pre><code> # From: # Handle a matched regex if (match := is not None: # Do something with match # A loop that can't be trivially rewritten using 2-arg iter() while chunk := process(chunk) # Reuse a value that's expensive to compute [y := f(x), y**2, y**3] # Share a subexpression between a comprehension filter clause and its output filtered_data = [y for x in data if (y := f(x)) is not None] </code></pre> <ul> <li>This article walks through trying this out with the 3.8 alpha’s now available.</li> <li>Using pyenv and brew to install 3.8, but you can also just download it and try it out. <ul> <li>3.8.0a1: <a href=""></a></li> <li>3.8.0a2: <a href=""></a></li> </ul></li> <li>Ends with a demonstration of the walrus operator working in a (I think) very likely use case, grabbing a value from a dict if the key exists</li> </ul> <pre><code> for entry in sample_data: if title := entry.get("title"): print(f'Found title: "{title}"') </code></pre> <ul> <li>That code won’t fail if the <code>title</code> key doesn’t exist.</li> </ul> <pstrong>Michael #4:</strong> <a href=""><strong>bullet : Beautiful Python Prompts Made Simple</strong></a></p> <ul> <li>Have you ever wanted a dropdown select box for your CLI? Bullet!</li> <li>Lots of design options</li> <li>Also <ul> <li>Password “boxes”</li> <li>Yes/No</li> <li>Numbers</li> </ul></li> <li>Looking for contributors, especially Windows support.</li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>Hosting private pip packages using Azure Artifacts</strong></a></p> <ul> <li>Interesting idea to utilize artifacts as a private place to store built packages to pip install elsewhere.</li> <li>Walkthrough is assuming you are working with a data pipeline.</li> <li>You can package some of the work in earlier stages for use in later stages by packaging them and making them available as artifacts.</li> <li>Includes a basic tutorial on setuptools packaging and building an sdist and a wheel.</li> <li>Need to use CI in the Azure DevOps tool and use that to build the package and save the artifact</li> <li>Now in a later stage where you want to install the package, there are some configs needed to get the pip credentials right, included in the article.</li> <li>Very fun article/hack to beat Azure into a use model that maybe it wasn’t designed for.</li> <li><p>Could be useful for non data pipeline usage, I’m sure.</p></li> <li><p>Speaking of Azure, we brought up Anthony Shaw’s <a href=""><strong>pytest-azurepipelines</strong></a> pytest plugin last week. Well, it is now part of the recommended <a href="">Python template from Azure</a>. Very cool.</p></li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>Async/await for wxPython</strong></a></p> <ul> <li>via Andy Bulka</li> <li>Remember asyncio and PyQt from last week?</li> <li>Similar project called wxasync which does the same thing for wxPython!</li> <li>He’s written a medium article about it with links to that project, and share some real life usage scenarios and fun demo apps.</li> <li>wxPython is important because it's free, even for commercial purposes (unlike PyQt).</li> <li>His article even contains a slightly controversial section entitled "Is async/await an anti-pattern?" which refers to the phenomenon of the async keyword potentially spreading through one's codebase, and some thoughts on how to mitigate that.</li> </ul> <pstrong>Extras</strong></p> <p><strong>Michael: Mongo license followup</strong></p> <ul> <li>Will S. told me I was wrong! <a href="">And I was</a>. :)</li> <li>The main clarification I wanted to make above was that the AGPL has been around for a while, and it is the new SSPL from MongoDB that targets cloud providers.</li> <li>Also, one other point I didn't mention -- the reason the SSPL isn't considered open source is that it places additional conditions on providing the software as a service and the OSI's open source definition requires no discrimination based on field of endeavor.</li> </ul> <p><strong>Michael:</strong> <code>python2</code> <strong>becomes self-aware, enters fifth stage of grief</strong></p> <ul> <li>Funny <a href="">thread I started</a></li> </ul> <blockquote> <p>python2 -m pip list DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.</p> </blockquote> <p>Michael: <a href="">PyDist — Simple Python Packaging</a></p> <ul> <li>Your private and public dependencies, all in one place.</li> <li>Looks to be paid, but with free beta?</li> <li>It mirrors the public PyPI index, and keeps packages and releases that have been deleted from PyPI. It allows organizations to upload their own private dependencies, and seamlessly create private forks of public packages. And it integrates with standard Python tools almost as well as PyPI does.</li> </ul> <pstrong>Joke</strong></p> <p>A metajoke: <code>pip install --user pyjokes</code> or even better <code>pipx install pyjokes</code>. Then:</p> <p><code>$ pyjoke</code></p> <p>[hilarity ensues! …]</p>
Mar 16, 2019
#120 AWS, MongoDB, and the Economic Realities of Open Source and more
<p>Sponsored by <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>The Ultimate Guide To Memorable Tech Talks</strong></a></p> <ul> <li>Nina Zakharenko</li> <li>7 part series that covers choosing a topic, writing a talk proposal, tools, planning, writing, practicing, and delivering the talk</li> <li>I’ve just read the tools section, and am looking forward to the rest of the series. <ul> <li>From the tools section: “I noticed I’d procrastinate on making the slides look good instead of focusing my time on making quality content.”</li> </ul></li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Running Flask on Kubernetes</strong></a></p> <ul> <li>via &amp; Michael Herman</li> <li>What is Kubernetes?</li> <li>A step-by-step tutorial that details how to deploy a Flask-based microservice (along with Postgres and Vue.js) to a Kubernetes cluster.</li> <li>Goals of tutorial <ol> <li>Explain what container orchestration is and why you may need to use an orchestration tool</li> <li>Discuss the pros and cons of using Kubernetes over other orchestration tools like Docker Swarm and Elastic Container Service (ECS)</li> <li>Explain the following Kubernetes primitives - Node, Pod, Service, Label, Deployment, Ingress, and Volume</li> <li>Spin up a Python-based microservice locally with Docker Compose</li> <li>Configure a Kubernetes cluster to run locally with Minikube</li> <li>Set up a volume to hold Postgres data within a Kubernetes cluster</li> <li>Use Kubernetes Secrets to manage sensitive information</li> <li>Run Flask, Gunicorn, Postgres, and Vue on Kubernetes</li> <li>Expose Flask and Vue to external users via an Ingress</li> </ol></li> </ul> <pstrong>Brian #3</strong>: <strong>Changes in the CI landscape</strong></p> <ul> <li><a href="">Travis CI joins the Idera family</a> - TravisCI blog</li> <li><a href=";ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1098583889864478720&amp;">#travisAlums</a> on Twitter <ul> <li><a href="">“TravisCI is laying off a bunch of senior engineers and other technical staff. Look at the #travisAlums hashtag and hire them!”</a> - alicegoldfuss</li> </ul></li> <li>options: <a href="">GitHub lists 17 options for CI</a>, including GitLab &amp; Azure Pipelines <ul> <li>Some relevant articles, resources: <ul> <li><a href="">The CI/CD market consolidation</a> - GitLab article</li> <li><a href="">Azure Pipelines with Python — by example</a> - Anthony Shaw</li> <li><a href="">pytest-azurepipelines</a> - Anthony Shaw</li> <li><a href="">Azure Pipelines Templates</a> - Anthony Sottile</li> </ul></li> </ul></li> </ul> <pstrong>Michael #4:</strong> <a href="">Python server setup for macOS 🍎</a></p> <ul> <li><strong>what</strong>: hello world for Python server setup on macOS </li> <li><strong>why</strong>: most guides show setup on a Linux server (which makes sense) but macoS is useful for learning and for local dev</li> <li>STEP 1: NGINX ➡️ STATIC ASSETS</li> <li>STEP 2: GUNICORN ➡️ FLASK</li> <li>STEP 3: NGINX ➡️ GUNICORN</li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>Learn Enough Python to be Useful: argparse</strong></a> </p> <ul> <li>How to Get Command Line Arguments Into Your Scripts - Jeff Hale</li> <li>“argparse is the “recommended command-line parsing module in the Python standard library.” It’s what you use to get command line arguments into your program.</li> <li>“I couldn’t find a good intro guide for argparse when I needed one, so I wrote this article.”</li> </ul> <pstrong>Michael #6:</strong> <a href="">AWS, MongoDB, and the Economic Realities of Open Source</a></p> <ul> <li>Related podcast:</li> <li><p>Last week, from the <a href="">AWS blog</a>:</p> <blockquote> <p>Today we are launching <a href="">Amazon DocumentDB (with MongoDB compatibility)</a>, a fast, scalable, and highly available document database that is designed to be compatible with your existing MongoDB applications and tools. Amazon DocumentDB uses a purpose-built SSD-based storage layer, with 6x replication across 3 separate Availability Zones. The storage layer is distributed, fault-tolerant, and self-healing, giving you the the performance, scalability, and availability needed to run production-scale MongoDB workloads.</p> </blockquote></li> <li><p>Like an increasing number of such projects, MongoDB is open source…or it was anyways. MongoDB Inc., a venture-backed company that IPO’d in October, 2017, made its core database server product available under the <a href="">GNU Affero General Public License (AGPL)</a>.</p></li> <li>AGPL extended the GPL to apply to software accessed over a network; since the software is only being used, not copied</li> <li>MongoDB’s Business Model</li> <li>We believe we have a highly differentiated business model that combines the developer mindshare and adoption benefits of open source with the economic benefits of a proprietary software subscription business model.</li> <li><ul> <li>MongoDB enterprise and MongoDB atlas</li> </ul></li> <li>Basically, MongoDB sells three things on top of its open source database server: <ul> <li>Additional tools for enterprise companies to implement MongoDB</li> <li>A hosted service for smaller companies to use MongoDB</li> <li>Legal certainty</li> </ul></li> <li>What AWS Sells</li> <li>the value of software is typically realized in three ways: <ul> <li>First is hardware. </li> <li>Second is licenses. This was Microsoft’s core business for decades: licenses sold to OEMs (for the consumer market) or to companies directly (for the enterprise market). </li> <li>Third is software-as-a-service.</li> </ul></li> <li>AWS announced last week: &gt; The storage layer is distributed, fault-tolerant, and self-healing, giving you the the performance, scalability, and availability needed to run production-scale MongoDB workloads.</li> <li>AWS is not selling MongoDB: what they are selling is “performance, scalability, and availability.” DocumentDB is just one particular area of many where those benefits are manifested on AWS.</li> <li>Thus we have arrived at a conundrum for open source companies: <ul> <li>MongoDB leveraged open source to gain mindshare.</li> <li>MongoDB Inc. built a successful company selling additional tools for enterprises to run MongoDB.</li> <li>More and more enterprises don’t want to run their own software: they want to hire AWS (or Microsoft or Google) to run it for them, because they value performance, scalability, and availability.</li> </ul></li> <li>This leaves MongoDB Inc. not unlike the record companies after the advent of downloads: what they sold was not software but rather the tools that made that software usable, but those tools are increasingly obsolete as computing moves to the cloud. And now AWS is selling what enterprises really want.</li> <li>This tradeoff is inescapable, and it is fair to wonder if the golden age of VC-funded open source companies will start to fade (although not open source generally). The monetization model depends on the friction of on-premise software; once cloud computing is dominant, the economic model is much more challenging.</li> </ul> <pstrong>Extras:</strong></p> <p><a href="">PyTexas</a> 2019 at <a href="">#Austin</a> on Apr 13th and 14th. Registrations now open. More info at <a href=""></a></p> <p>Michael: Sorry Ant!</p> <p>Michael: RustPython follow up: <a href=""></a>demo/</p> <pstrong>Joke:</strong></p> <ul> <li><strong>Q:</strong> Why was the developer unhappy at their job?</li> <li><p><strong>A:</strong> They wanted arrays.</p></li> <li><p><strong>Q:</strong> Where did the parallel function wash its hands?</p></li> <li><strong>A:</strong> Async</li> </ul>
Mar 05, 2019
#119 Assorted files as Django ORM backends with Alkali
<p>Sponsored by <a href=""><strong></strong></a></p> <p><strong>Special guests</strong></p> <ul> <li><a href="">Eric Chou</a></li> <li><a href="">Dan Bader</a></li> <li><a href="">Trey Hunner</a></li> </ul> <pstrong>Michael #1:</strong> <a href=""><strong>Incrementally migrating over one million lines of code from Python 2 to Python 3</strong></a></p> <ul> <li>Weighing in at over 1 million lines of Python logic, we had a massive surface area for potential issues in our migration from Python 2 to Python 3</li> <li>First Py3 commit, hack week 2015 <ul> <li>Unfortunately, it was clear that many features were completely broken by the upgrade</li> </ul></li> <li>Official start H1 2017</li> <li>Armed with <a href="">Mypy</a>, a static type-checking tool that we had adopted in the interim year, they made substantial strides towards enabling the Python 3 migration: <ul> <li>Ported our custom fork of Python to version 3.5</li> <li>Upgraded some Python dependencies to Python 3-compatible versions, and forked some others (e.g. <code>babel</code>)</li> <li>Modified some Dropbox client code to be Python 3 compatible</li> <li>Set up automated jobs in our continuous integration (CI) to run the existing unit tests with the Python 3 interpreter, and Mypy type-checking in Python 3 mode</li> </ul></li> <li>Crucially, the automated tests meant that we could be certain that the limited Python 3 compatibility that existed would not have regressed when the project was picked up again.</li> <li><strong>Prerequisites</strong></li> <li>Before we could begin working on migrating any of our application logic, we had to ensure that we could load the Python 3 interpreter and run until the entry point of the application. In the past, we had used “freezer” scripts to do this for us. However, none of these had support for Python 3 around this time, so in late 2016, we built a custom, more native solution which we internally referred to as “Anti-freeze” (more on that in <a href="">the initial Python 3 migration blog post</a>).</li> <li>Incrementally enabling unit tests and type-checking</li> <li>‘Straddling’ Python 2 and Python 3</li> <li>Letting it bake</li> <li>Learnings (tl;dr) <ul> <li>Unit tests and typing are invaluable.</li> <li>String encoding in Python is hard.</li> <li>Incrementally migrate to Python 3 for great profit.</li> </ul></li> </ul> <pstrong>Eric #2:</strong> <strong>Network Automation Development with Python (for fun and for profit)</strong></p> <ul> <li>Terms: NetDevOps (Cisco), NRE (Network Reliability Engineer)</li> <li>Libraires: <a href="">Netmiko</a>, <a href="">NAPALM</a>, <a href="">Nornir</a> </li> <li>Free Lab Resources: <a href="">NRE Labs</a>, <a href="">dCloud</a>, <a href="">DevNet</a></li> <li>Conferences: AnsibleFest (network automation track), <a href="">Cisco DevnetCreate</a></li> </ul> <pstrong>Trey #3</strong>: <a href="">Alkali file as DB</a></p> <ul> <li>If you have structured data you want to query (like RSS feed, CSV, JSON, or any custom format of your own creation) you can use a Django ORM-like syntax to query it</li> <li>Save it to the same format or a different format because you control both the reading and the writing</li> <li>Kurt is at PyCascades so I got to chat with him about this</li> </ul> <pstrong>Dan #4:</strong> <a href=""><strong>Carnegie Mellon Launches Undergraduate Degree in Artificial Intelligence</strong></a> <em>*</em>*</p> <ul> <li>Carnegie Mellon University's School of Computer Science will offer a new <a href="">undergraduate degree in artificial intelligence</a> beginning this fall</li> <li>The first offered by a U.S. university</li> <li>"Specialists in artificial intelligence have never been more important, in shorter supply or in greater demand by employers," said Andrew Moore, dean of the School of Computer Science.</li> <li>The bachelor's degree in AI will focus more on how complex inputs — such as vision, language and huge databases — are used to make decisions or enhance human capabilities</li> </ul> <pstrong>Michael #5:</strong> <a href=""><strong>asyncio + PyQt5/PySide2</strong></a></p> <ul> <li>via <a href="">Florian Dahlitz</a></li> <li><code>asyncqt</code> is an implementation of the <code>PEP 3156</code> event-loop with Qt. </li> <li>This package is a fork of <code>quamash</code> focusing on modern Python versions, with some extra utilities, examples and simplified CI.</li> <li>Allows wiring events to Qt’s event loop that run on asyncio and leverage it internally.</li> <li>Example: <a href=""></a></li> </ul> <pstrong>Dan #6:</strong> <a href=""><strong>4 things I want to see in Python 4.0</strong></a></p> <ol> <li>JIT as a first class feature</li> <li>A stable .0 release</li> <li>Static type hinting</li> <li>A GPU story for multiprocessing</li> <li>More community contributions</li> </ol> <pstrong>Extras:</strong></p> <p>Michael: My Python Async webcast <a href=""><strong>recording is now available</strong></a>. Michael: PyCon Israel in the first week of June (<a href=""></a>), and the CFP opened today: <a href=""></a> Dan: <a href="">Python Basics Book</a></p> <pstrong>Joke:</strong></p> <ul> <li><strong>Q:</strong> Why did the developer ground their kid?</li> <li><strong>A:</strong> They weren't telling the <strong>truthy</strong></li> </ul>
Feb 26, 2019
#118 Better Python executable management with pipx
<p>Sponsored by <a href=""><strong></strong></a></p> <pstrong>Brian #1</strong>: <a href=""><strong>Frozen-Flask</strong></a></p> <ul> <li>“Frozen-Flask freezes a <a href="">Flask</a> application into a set of static files. The result can be hosted without any server-side software other than a traditional web server.”</li> <li>2012 tutorial, <a href=""><strong>Dead easy yet powerful static website generator with Flask</strong></a></li> <li>Some of it is out of date, but it does point to the power of Frozen-Flask, as well as highlight a cool plugin, <a href="">Flask-FlatPages</a>, which allows pages from markdown.</li> </ul> <pstrong>Michael #2</strong>: <a href=""><strong>pipx</strong></a></p> <ul> <li>by Chad Smith</li> <li>Last week we spoke about <a href="">pythonloc</a></li> <li>Execute binaries from Python packages in isolated environments</li> <li><em>"binary" to describe a CLI application that can be run directly from the command line</em></li> <li>Features <ul> <li>Safely install packages to isolated virtual environments, while globally exposing their CLI applications so you can run them from anywhere</li> <li>Easily list, upgrade, and uninstall packages that were installed with pipx</li> <li>Run the latest version of a CLI application from a package in a temporary virtual environment, leaving your system untouched after it finishes</li> <li>Run binaries from the <code>__pypackages__</code> directory per PEP 582 as companion tool to <a href="">pythonloc</a></li> <li>Runs with regular user permissions, never calling <code>sudo pip install ...</code> (you aren't doing that, are you? 😄).</li> </ul></li> <li>You can globally install a CLI application by running: <code>pipx install PACKAGE</code></li> <li>"Just the “pipx upgrade-all” command is already a huge win over pipsi"</li> <li>Check out <a href="">How does this compare to pipsi?</a></li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>Data science is different now</strong></a></p> <ul> <li>Vicki Boykis</li> <li>There’s lots of buzz around data science.</li> <li>This has resulted in loads of new data scientists looking for junior level positions. <ul> <li>Coming from boot camps, MOOCs, self taught, remote degrees, and other training.</li> </ul></li> <li>“.. now that data science has changed from a buzzword to something even larger companies outside of the Silicon Valley bubble hire for, positions have not only become more codified, but with more rigorous entry requirements that will prefer people with previous data science experience every time.”</li> <li>“ … the market can be very hard, and very discouraging for the flood of beginners.”</li> <li>Data science is a misleading job req <ul> <li>“The reality is that “data science” has never been as much about machine learning as it has about cleaning, shaping data, and moving it from place to place.”</li> </ul></li> <li>Advice: <ul> <li>Don’t get into data science (this amuses me).</li> <li>“Don’t do what everyone else is doing, because it won’t differentiate you.” <ul> <li>“It’s much easier to come into a data science and tech career through the “back door”, i.e. starting out as a junior developer, or in DevOps, project management, and, perhaps most relevant, as a data analyst, information manager, or similar, than it is to apply point-blank for the same 5 positions that everyone else is applying to. It will take longer, but at the same time as you’re working towards that data science job, you’re learning critical IT skills that will be important to you your entire career.”</li> </ul></li> <li>Learn the skills needed for data science today <ul> <li>Creating Python packages</li> <li>Putting R in production</li> <li>Optimizing Spark jobs so they run more efficiently</li> <li>Version controlling data</li> <li>Making models and data reproducible</li> <li>Version controlling SQL</li> <li>Building and maintaining clean data in data lakes</li> <li>Tooling for time series forecasting at scale</li> <li>Scaling sharing of Jupyter notebooks</li> <li>Thinking about systems for clean data</li> <li>Lots of JSON</li> </ul></li> </ul></li> <li>Data science is turning more and more into a mostly engineering field.</li> <li>Data scientists need to have “good generalist engineering skills with a data background.”</li> </ul> <pstrong>Michael #4</strong>: <a href=""><strong>RustPython</strong></a></p> <ul> <li>via <a href="">Fredrik Averpil</a></li> <li>A Python-3 (CPython &gt;= 3.5.0) Interpreter written in Rust.</li> <li>Seems pretty active: Latest commit ac95b61 an hour ago…</li> <li>Goals <ul> <li>Full Python-3 environment entirely in Rust (not CPython bindings)</li> <li>A clean implementation without compatibility hacks</li> </ul></li> <li>Contributing <ul> <li>To start contributing, there are a lot of things that need to be done.</li> <li>Most tasks are listed in the issue tracker. Check issues labeled with good first issue if you wish to start coding.</li> </ul></li> <li>Rust does have direct WebAssembly support…</li> </ul> <pstrong>Brian #5</strong>: <a href=""><strong>Jupyter Notebook: An Introduction</strong></a></p> <ul> <li>Mike Driscoll on RealPython</li> <li>Not the “all the cool things you can do with it”, but the “really, how do I start” tutuorial. <ul> <li>I think it should have included a mention of installing it in a venv and how to use <code>%pip install</code>, so I’ll include those things in these notes.</li> </ul></li> <li>Installing with <code>pip install jupyter</code> . <ul> <li>Also a note that Jupyter is included with the Anaconda distribution.</li> <li>Note: Like everything else, I always install it in a virtual environment, if using <code>pip</code>, so the real installation instructions I recommend is: <ul> <li><code>python3 -m venv venv</code> <code>--``prompt jupyter</code></li> <li><code>source venv/bin/activate</code> OR <code>venv\scripts\activate.bat</code> if windows</li> <li><code>pip install jupyter</code></li> <li><code>pip install [HTML_REMOVED]</code></li> <li><code>jupyter notebook</code></li> <li>That will launch a localhost web interface.</li> </ul></li> </ul></li> <li>Creating a new notebook within the web interface.</li> <li>Changing the “Untitled” name by clicking on the name. <ul> <li>This was not obvious to me.</li> </ul></li> <li>Running cells, including the shift-enter keyboard shortcut.</li> <li>A run through the menu, stopping at non-obvious places <ul> <li>“File” has “Save and Checkpoint” which is super cool.</li> <li>“Edit” has cell cut, copy, paste. But also has delete, split, merge, and cell movement.</li> <li>“Cell” menu has lots of cool run options, like “Run all above” and “Run all below” and others.</li> </ul></li> <li>Not just Python, but you can have a terminal sessions and more from within Jupyter.</li> <li>A look at the “Running” tab.</li> <li>Quick overview of the markdown support for markdown cells</li> <li><p>Exporting notebooks using <code>jupyter nbconvert</code></p></li> <li><p>Extra notes on installing packages from Jupyter:</p> <ul> <li>To pip install from the notebook, do this: <code>%pip install numpy</code> in a code cell.</li> </ul></li> </ul> <pstrong>Michael #6</strong>: <a href=""><strong>Python Developers Survey 2018 Results</strong></a></p> <ul> <li>Python usage as a main language is up 5 percentage points from 79% in 2017 when Python Software Foundation conducted its previous survey.</li> <li><strong>What do you use Python for? (2018/2017)</strong> <ul> <li>59%/51% Data analysis</li> <li>56%/54% Web dev</li> <li>39%/32% ML</li> <li>Web development is the only category with a large gap (56% vs. 36%) separating those using Python as their main language vs. as a supplementary language. For other types of development, the differences are much smaller.</li> </ul></li> <li><strong>What do you use Python for the most? (single answer)</strong> <ul> <li>29%/29% web dev</li> <li>17%/17% data analysis</li> <li>11%/8% ML</li> </ul></li> <li><strong>Like last year</strong>: <ul> <li>27% (Web development) ≈ 28% (Scientific development) <ul> <li>Science = 17% + 11% for Data analysis + Machine learning</li> </ul></li> </ul></li> <li><strong>Python 3 vs Python 2</strong> <ul> <li>84% Python 3 vs 16% Python 2. The use of Python 3 continues to grow rapidly. According to the latest research in 2017, 75% were using Python 3 compared with 25% for Python 2.</li> </ul></li> <li><strong>Top 4 web frameworks</strong> (majority to the first two): <ul> <li>Flask</li> <li>Django</li> <li>Tornado</li> <li>Pyramid</li> </ul></li> <li><strong>Databases</strong> <ul> <li>PostgreSQL</li> <li>MySQL</li> <li>SQLite</li> <li>MongoDB</li> </ul></li> <li>ORMs <ul> <li>SQLAlchemy and Django ORM tied</li> </ul></li> </ul> <pstrong>Extras</strong>:</p> <ul> <li>“Mentored sprints for diverse beginners” at PyCon <ul> <li>“<strong>A newcomer’s introduction to contributing to an open source project”</strong></li> <li></li> <li>Call for applications for projects open Feb 8 to March 14</li> <li>Call for contributors, participants in the sprint also open Feb 8 to March 14</li> <li>“<strong>If you are wondering if this event is for you: it definitely is and we would love to have you taking part in this sprint.”</strong></li> <li>“This mentored sprint will take place on Saturday, May 4th, 2019 from 2:35pm to 6:30pm”</li> </ul></li> </ul> <pstrong>Joke</strong>: via Florian Q: If you have some pseudo code (say in sample.txt) how do you most easily convert it to Python? A: Change the extension to .py </p> <p>Extra Joke: <a href=""><strong>Python Song (with chapters!)</strong></a></p>
Feb 22, 2019
#117 Is this the end of Python virtual environments?
<p>Sponsored by <a href=""><strong></strong></a></p> <pstrong>Brian #1</strong>: <a href=""><strong>Goodbye Virtual Environments?</strong></a></p> <ul> <li>by <a href="">Chad Smith</a></li> <li>venv’s are great but they introduce some problems as well: <ul> <li><strong>Learning curve:</strong> explaining “virtual environments” to people who just want to jump in and code is not always easy</li> <li><strong>Terminal isolation:</strong> Virtual Environments are activated and deactivated on a per-terminal basis</li> <li><strong>Cognitive overhead:</strong> Setting up, remembering installation location, activating/deactivating</li> </ul></li> <li><a href="">PEP 582 — Python local packages directory</a> <ul> <li>This PEP proposes to add to Python a mechanism to automatically recognize a <code>__pypackages__</code>directory and prefer importing packages installed in this location over user or global site-packages. This will avoid the steps to create, activate or deactivate “virtual environments”. Python will use the <code>__pypackages__</code> from the base directory of the script when present.</li> </ul></li> <li>Try it now with <a href=""><strong>pythonloc</strong></a> <ul> <li><strong>pythonloc</strong> is a drop in replacement for <code>python</code> and <code>pip</code> that automatically recognizes a <code>__pypackages__</code> directory and prefers importing packages installed in this location over user or global site-packages. If you are familiar with node, <code>__pypackages__</code> works similarly to <code>node_modules</code>.</li> <li>Instead of running <code>python</code> you run <code>pythonloc</code> and the <code>__pypackages__</code> path will automatically be searched first for packages. And instead of running <code>pip</code> you run <code>piploc</code> and it will install/uninstall from <code>__pypackages__</code>.</li> </ul></li> </ul> <pstrong>Michael #2</strong>: <a href=""><strong>webassets</strong></a></p> <ul> <li>Bundles and minifies CSS &amp; JS files</li> <li>Been doing a lot of work to rank higher on the sites</li> <li>That lead me to <a href=""><strong>Google’s Lighthouse</strong></a></li> <li>Despite 25ms response time to the network, Google thought my site was “kinda slow”, yikes!</li> <li>webassets has integration for the big three: Django, Flask, &amp; Pyramid. <ul> <li>But I prefer to just generate them and serve them off disk</li> </ul></li> </ul> <pre><code> def build_asset(env: webassets.Environment, files: List[str], filters: str, output: str): bundle = webassets.Bundle( *files, filters=filters, output=output, env=env ) </code></pre> <pstrong>Brian #3</strong>: <strong>Bernat on Python Packaging</strong></p> <ul> <li>3 part series by Bernat Gabor <ul> <li>Maintainer of tox and virtualenv Python packages.</li> </ul></li> <li><a href="">The State of Python Packaging</a></li> <li><a href="">Python packaging - Past, Present, Future</a></li> <li><a href="">Python packaging - Growing Pains</a></li> </ul> <pstrong>Michael #4</strong>: <a href=""><strong>What the mock? — A cheatsheet for mocking in Python</strong></a></p> <ul> <li>Nice introduction</li> <li>Some examples</li> </ul> <pre><code> @mock.patch('work.os') def test_using_decorator(self, mocked_os): work_on() mocked_os.getcwd.assert_called_once() </code></pre> <p>And</p> <pre><code> def test_using_context_manager(self): with mock.patch('work.os') as mocked_os: work_on() mocked_os.getcwd.assert_called_once() </code></pre> <pstrong>Brian #5</strong>: <a href=""><strong>Transitions: The easiest way to improve your tech talk</strong></a></p> <ul> <li>By Saron Yitbarek</li> <li>Jeff Atwood of CodingHorror noted “The people who can write and communicate effectively are, all too often, the only people who get heard. They get to set the terms of the debate.”</li> <li>Effectively presenting is part of effective communication.</li> <li>I love the focus of this article. Focused on one little aspect of improving the performance of a tech talk.</li> </ul> <pstrong>Michael #6</strong>: <a href=""><strong>Steering council announced</strong></a></p> <ul> <li>Our new leaders are <ul> <li>Barry Warsaw</li> <li>Brett Cannon</li> <li>Carol Willing</li> <li>Guido van Rossum</li> <li>Nick Coghlan</li> </ul></li> <li>Via Joe Carey</li> <li>We both think it’s great Guido is on the council.</li> </ul> <pxtras: </p> <ul> <li>Brian: Got interviewed on IT Energizer Podcast: <ul> <li>The one with Brian: <a href=""></a></li> <li>The one with Michael: <a href=""></a></li> </ul></li> <li><a href="">PyCon LATAM</a> <ul> <li>August 29, Puerto Vallarta, Mexico</li> <li>We should go. Anyone want to sponsor our travel/hotel to this event?</li> <li>CFP open till May 31, 2019, <a href=""></a></li> </ul></li> </ul> <poke:</p> <p>From the list from Ant, my votes. </p> <ul> <li><p><strong>Q:</strong> What's the second movie about a database engineer called? <strong>A:</strong> The SQL.</p></li> <li><p><strong>!false</strong> It's funny 'cause it's true.</p></li> <li><p>A programmer's spouse tells them, "Run to the store and pick up a loaf of bread. If they have eggs, get a dozen." The programmer comes home with 12 loaves of bread.</p></li> </ul>
Feb 14, 2019
#116 So you want Python in a 3D graphics engine?
<p>Sponsored by <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>Inside python dict — an explorable explanation</strong></a></p> <ul> <li>Interactive tutorial on dictionaries <ul> <li>Searching efficiently in a list</li> <li>Why are hash tables called has tables?</li> <li>Putting it all together to make an “almost”-Python-dict</li> <li>How Python dict really works internally</li> </ul></li> <li>Yes this is a super deep dive, but wow it’s cool.</li> <li>Tons of the code is runnable right there in the web page, including moving visual representations, highlighted code with current line of code highlighted.</li> <li>Some examples allow you to edit values and play with stuff.</li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Embed Python in Unreal Engine 4</strong></a> </p> <ul> <li>You may notice a theme throughout my set of picks on this episode</li> <li>Games built on Unreal Engine 4 include <ul> <li><a href=""><em>Fortnite: Save the World</em></a></li> <li><a href=""><em>Gears of War 4</em></a></li> <li><a href=""><em>Marvel vs. Capcom: Infinite</em></a></li> <li><a href=""><em>Moto Racer 4</em></a></li> <li><a href=""><em>System Shock (remake)</em></a></li> </ul></li> <li>Plugin embedding a whole Python VM in Unreal Engine 4 (both the editor and runtime).</li> <li>This means you can use the plugin to write other plugins, to automate tasks, to write unit tests and to implement gameplay elements.</li> <li>Here is an <a href="">example usage</a>. It’s a really nice overview and tutorial for the editor.</li> <li>For game elements, check out <a href="">this section</a>.</li> </ul> <pstrong>Brian #3: Redirecting stdout with contextlib</strong></p> <ul> <li>When I want to test the stdout output of some code, that’s easy, I grab the <a href="">capsys fixture</a> from pytest.</li> <li>But what if you want to grab the stdout of a method NOT while testing?</li> <li>Enter <code>[contextlib.redirect_stdout(new_target)](</code></li> <li>so cool. And very easy to read.</li> <li>ex:</li> </ul> <pre><code> f = io.StringIO() with redirect_stdout(f): help(pow) s = f.getvalue() </code></pre> <ul> <li>also a version for <code>stderr</code></li> </ul> <pstrong>Michael #4:</strong> <a href="">Panda3D</a></p> <ul> <li>via Kolja Lubitz</li> <li>Panda3D is an open-source, completely free-to-use engine for realtime 3D games, visualizations, simulations, experiments</li> <li>Not just games, could be science as well!</li> <li>The full power of the graphics card is exposed through an easy-to-use API. Panda3D combines the speed of C++ with the ease of use of Python to give you a fast rate of development without sacrificing on performance.</li> <li>Features: <ul> <li><strong>Platform Portability</strong></li> <li><strong>Flexible Asset Handling</strong>: Panda3D includes command-line tools for processing and optimizing source assets, allowing you to automate and script your content production pipeline to fit your exact needs.</li> <li><strong>Library Bindings</strong>: Panda3D comes with out-of-the-box support for many popular third-party libraries, such as the Bullet physics engine, Assimp model loader, OpenAL</li> <li><strong>Performance Profiling</strong>: Panda3D includes pstats — an over-the-network profiling system designed to help you understand where every single millisecond of your frame time goes.</li> </ul></li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>Why PyPI Doesn't Know Your Projects Dependencies</strong></a></p> <ul> <li>Some questions you may have asked: &gt; How can I produce a dependency graph for Python packages? &gt; Why doesn’t PyPI show a project’s dependencies on it’s project page? &gt; How can I get a project’s dependencies without downloading the package? &gt; Can I search PyPI and filter out projects that have a certain dependency?</li> <li>If everything is in <code>requirements.txt</code>, you just might be able to, but…</li> <li><code></code> is dynamic. You gotta run it to see what’s needed.</li> <li>Dependencies might be environment specific. Windows vs Linux vs Mac, as an example.</li> <li>Nothing stopping someone from putting <code>random.choice()</code> for dependencies in a <code></code> file. But that would be kinda evil. But could be done. (Listener homework?)</li> <li>The <code>wheel</code> format is way more predictable because it limits some of this freedom. <code>wheel</code>s don’t get run when they install, they really just get unpacked.</li> <li>More info on wheels: Kind of a tangent, but what why not: <ul> <li>From: <a href=""></a></li> <li>“<strong>Advantages of wheels</strong> <ul> <li>Faster installation for pure Python and native C extension packages.</li> <li>Avoids arbitrary code execution for installation. (Avoids</li> <li>Installation of a C extension does not require a compiler on Linux, Windows or macOS.</li> <li>Allows better caching for testing and continuous integration.</li> <li>Creates .pyc files as part of installation to ensure they match the Python interpreter used.</li> <li>More consistent installs across platforms and machines.”</li> </ul></li> </ul></li> </ul> <pstrong>Michael #6: PyGame series</strong></p> <ul> <li>via Matthew Ward</li> <li><a href="">Learn how to program in Python by building a simple dice game</a></li> <li><a href="">Build a game framework with Python using the PyGame module</a></li> <li><a href="">How to add a player to your Python game</a></li> <li><a href="">Using PyGame to move your game character around</a></li> <li><a href="">What's a hero without a villain? How to add one to your Python game</a></li> <li><a href="">Put platforms in a Python game with PyGame</a></li> <li>Also: Shout out to <a href="">Mission Python book: Code a Space Adventure Game!</a></li> </ul> <pxtras:</p> <poke (maybe, Brain feel free to pick another one):</p> <ul> <li>via <a href="">@realpython</a></li> <li>Why do Pythons live on land? They are above C-level!</li> </ul>
Feb 06, 2019
#115 Dataclass CSV reader and Nina drops by
<p>Sponsored by <a href=""><strong></strong></a></p> <p>Special guest: <a href="">Nina Z</a><a href="">akharenko</a></p> <pstrong>Brian #1:</strong> <a href=""><strong>Great Expectations</strong></a></p> <ul> <li>A set of tools intended for batch time testing of data pipeline data.</li> <li>Introduction to the problem doc: <a href="">Down with Pipeline debt / Introducing Great Expectations</a></li> <li><code>expect_[something]()</code> methods that return json formatted descriptions of whether or not the passed in data matches your expectations.</li> <li>Can be used programmatically or interactively in a notebook. (<a href="">video demo</a>).</li> <li>For programmatic use, I’m assuming you have to put code in place to stop a pipeline stage if expectations aren’t met, and write failing json result to a log or something.</li> <li>Examples, just a few, full list is big: <ul> <li>Table shape: <ul> <li>expect_column_to_exist, expect_table_row_count_to_equal</li> </ul></li> </ul></li> <li>Missing values, unique values, and types: - expect_column_values_to_be_unique, expect_column_values_to_not_be_null <ul> <li>Sets and ranges <ul> <li>expect_column_values_to_be_in_set</li> </ul></li> <li>String matching <ul> <li>expect_column_values_to_match_regex</li> </ul></li> <li>Datetime and JSON parsing</li> <li>Aggregate functions <ul> <li>expect_column_stdev_to_be_between</li> </ul></li> <li>Column pairs</li> <li>Distributional functions <ul> <li>expect_column_chisquare_test_p_value_to_be_greater_than</li> </ul></li> </ul></li> </ul> <pstrong>Nina #2:</strong> <strong>Using CircuitPython and MicroPython to write Python for wearable electronics and embedded platforms</strong></p> <ul> <li>I’ve been playing with electronics projects as a hobby for the past two years, and a few months ago turned my attention to Python on microcontrollers</li> <li><a href="">MicroPython</a> is a lean and efficient implementation of Python3 that can run on microcontrollers with just 256k of code space, and 16k of RAM. <a href="">CircuitPython</a> is a port of MicroPython, optimized for Adafruit devices. </li> <li>Some of the devices that run Python are as <a href="">small as a quarter</a>.</li> <li>My favorite Python hardware platform for beginners is <a href="">Adafruit’s Circuit PlayGround Express</a>. It has everything you need to get started with programming hardware without soldering. All you’ll need is alligator clips for the conductive pads. <ul> <li>The board features NeoPixel LEDs, buttons, switches, temperature, motion, and sound sensors, a tiny speaker, and lots more. You can even use it to control servos, tiny motor arms.</li> <li>Best of all, it only costs $25.</li> </ul></li> <li>If you want to program the Circuit PlayGround Express with a drag-n-drop style scratch-like interface, you can use <a href="">Microsoft’s MakeCode</a>. It’s perfect for kids and you’ll find lots of examples on their site.</li> <li>Best of all, there are tons of <a href="">guides for Python projects</a> to build on their website, from making your own synthesizers, to jewelry, to silly little robots. </li> <li>Check out the <a href="">repo</a> for my Python-powered earrings, see a <a href="">photo</a>, or a <a href="">demo</a>. </li> <li>Sign up for the Adafruit Python for Microcontrollers mailing list <a href="">here</a>, or see the archives <a href="">here</a>. </li> </ul> <pstrong>Michael #3</strong>: <a href=""><strong>Data class CSV reader</strong></a></p> <ul> <li>Map CSV to Data Classes </li> <li>You probably know about reading CSV files <ul> <li>Maybe as tuples</li> <li>Better with csv.DictReader</li> </ul></li> <li>This library is similar but maps Python 3.7’s data classes to rows of CSV files</li> <li>Includes type conversions (say string to int)</li> <li>Automatic type conversion. <code>DataclassReader</code> supports <code>str</code>, <code>int</code>, <code>float</code>, <code>complex</code> and <code>datetime</code></li> <li><code>DataclassReader</code> use the type annotation to perform validation of the data of the CSV file.</li> <li>Helps you troubleshoot issues with the data in the CSV file. <code>DataclassReader</code> will show exactly in which line of the CSV file contain errors.</li> <li>Extract only the data you need. It will only parse the properties defined in the <code>dataclass</code></li> <li>It uses <code>dataclass</code> features that let you define metadata properties so the data can be parsed exactly the way you want.</li> <li>Make the code cleaner. No more extra loops to convert data to the correct type, perform validation, set default values, the <code>DataclassReader</code> will do all this for you</li> <li>Default fallback values, more.</li> </ul> <pstrong>Brian #4:</strong> <a href=""><strong>How to Rock Python Packaging with Poetry and Briefcase</strong></a></p> <ul> <li>Starts with a discussion of the packaging (for those readers that don’t listen to Python Bytes, I guess.) However, it also puts flit, pipenv, and poetry in context with each other, which is nice.</li> <li>Runs through a tutorial of how to build a pyproject.toml based project using poetry and briefcase.</li> <li>We’ve talked about Poetry before, on <a href="">episode 100</a>.</li> <li>pyproject.toml is discussed extensively on <a href="">Test &amp; Code 52</a>.</li> <li><a href="">briefcase</a> is new, though, it’s a project for creating standalone native applications for Mac, Windows, Linux, iOS, Android, and more.</li> <li>The tutorial also discusses using poetry directly to publish to the test-pypi server. This is a nice touch. Use the test-pypi before pushing to the real pypi. Very cool.</li> </ul> <pstrong>Nina #5:</strong> <a href=""><strong>awesome-python-security</strong></a> <strong><em>*🕶🐍🔐</strong>, a collection of tools, techniques, and resources to make your Python more secure</em>*</p> <ul> <li>All of your production and client-facing code should be written with security in mind</li> <li>This list features a few resources I’ve heard of such as <a href="">Anthony Shaw’s excellent 10 common security gotchas article</a> which highlights problems like input injection and depending on assert statements in production, and a few that are new to me:</li> <li><a href="">OWASP (Open Web Application Security Project) Python Resources</a> at</li> <li><a href="">bandit</a> a tool to find common security issues in Python <ul> <li>bandit features a lot of useful plugins, that test for issues like: <ul> <li>hardcoded password strings</li> <li>leaving flask debug on in production</li> <li>using exec() in your code</li> <li><a href="">&amp; more</a></li> </ul></li> </ul></li> <li><a href="">detect-secrets</a>, a tool to detect secrets left accidentally in a Python codebase</li> <li>&amp; lots more like resources for learning about security concepts like cryptography</li> <li>See the <a href="">full list</a> for more</li> </ul> <pstrong>Michael #6: <a href="">pydbg</strong></a></p> <ul> <li>Python implementation of the Rust <code>dbg</code> macro</li> <li>Best seen with an example. Rather than printing things you want to inspect, you:</li> </ul> <pre><code> a = 2 b = 3 dbg(a+b) def square(x: int) -&gt; int: return x * x dbg(square(a)) </code></pre> <p>outputs:</p> <pre><code> [] a+b = 5 [] square(a) = 4 </code></pre> <pstrong>Extras:</strong></p> <p><strong>Brian:</strong></p> <ul> <li>pathlib + pytest tmpdir → tmp_path &amp; tmp_path_factory <ul> <li><a href=""></a></li> <li>These two new fixtures (as of pytest 3.9) act like the good old tmpdir and tmpdir_factory, but return pathlib Path objects. Awesome.</li> </ul></li> </ul> <p><strong>Michael:</strong> </p> <ul> <li><a href=""><em>The Art of Python</em></a> is a miniature arts festival at <a href="">PyCon North America 2019</a>, focusing on narrative, performance, and visual art. We intend to encourage and showcase novel art that helps us share our emotionally charged experiences of programming (particularly in Python). We hope that by attending, our audience will discover new aspects of empathy and rapport, and find a different kind of delight and perspective than might otherwise be expected at a large conference.</li> <li>StackOverflow Survey is Open! <a href=""></a> </li> <li><a href="">NumPy Is Awaiting Fix for Critical Remote Code Execution Bug</a> <ul> <li>via Doug Sheehan</li> <li>The issue was raised on January 16 and affects <a href="">NumPy</a> versions 1.10 (released in 2015) through 1.16, which is the latest release at the moment, released on January 14</li> <li>The problem is with the '<a href="">pickle</a>' module, which is used for transforming Python object structures into a format that can be stored on disk or in databases, or that allows delivery across a network.</li> <li>The issue was <a href="">reported</a> by security researcher <a href="">Sherwel Nan</a>, who says that if a Python application loads malicious data via the <code>numpy.load</code> function an attacker can obtain remote code execution on the machine.</li> </ul></li> <li>Get your google data <ul> <li>All google docs in MS Office format via <a href=""></a></li> <li>All Gmail in MBOX format from there as well</li> <li>Hint: Start with nothing selected ;)</li> </ul></li> </ul> <p><strong>Nina</strong>:</p> <ul> <li>I’m teaching a two day <a href="">Intro</a> and <a href="">Intermediate</a> Python course on March 19th and 20th. The class will live-stream for <strong>free</strong> <a href="">here</a> on each day of or join in-person from downtown Minneapolis. All of the course materials will be released for free as well.</li> <li>I recently recorded a series of videos with Carlton Gibson (Django maintainer) on developing Django Web Apps with VS Code, deploying them to Azure with a few clicks, setting up a Continuous Integration / Continuous Delivery pipeline, and creating serverless apps. Watch the series here: <a href=""></a></li> <li>I’ll be a mentor at a brand new hatchery event at PyCon US 2019, mentored sprints for diverse beginners organized by Tania Allard. The goal is to help underrepresented folks at PyCon contribute to open source in a supportive environment. The details will be located <a href="">here</a> (currently a placeholder) when they’re finalized.</li> <li>Catch my talk about <a href="">electronics projects in Python with LEDs at PyCascades</a> in Seattle on February 24th. Currently tickets are still for sale. </li> <li>If you haven’t tried the <a href="">Python extension for VS Code</a>, now is a great time. The <a href="">December release</a> included some killer features, such as remote Jupyter support, and exporting Python files as Jupyter notebooks. Keep up with future releases at the <a href="">Python at Microsoft blog</a>. </li> </ul> <pokes:</p> <ul> <li>Q: What do you call a snake that only eats desert? A: A pie-thon. (might not make sense read out loud)</li> <li>Q: How do you measure a python? A: In inches. They don't have any feet! </li> <li>Q: What is a python’s favorite subject? Hiss-tory!</li> </ul>
Feb 02, 2019
#114 What should be in the Python standard library?
<p>Sponsored by <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>What should be in the Python standard library?</strong></a></p> <ul> <li>on by Jake Edge</li> <li>There was a discussion recently about what should be in the standard library, triggered by a request to add LZ4 compression.</li> <li>Kinda hard to summarize but we’ll try: <ul> <li>Jonathan Underwood proposed adding LZ4 compression to stdlib.</li> <li>Can of worms opened</li> <li>zlib and bz2 already in stdlib</li> <li>Brett proposed making something similar to hashlib for compression algorithms.</li> <li>Against adding it: <ul> <li>lz4 not needed for stdlib, and actually, bz2 isn’t either, but it’s kinda late to remove.</li> </ul></li> <li>PyPI is easy enough. put stuff there.</li> <li>Led to a discussion of the role of stdlib. <ul> <li>If it’s batteries included, shouldn’t we add new batteries</li> <li>Some people don’t have access to PyPI easily</li> <li>Do we never remove elements? really?</li> <li>Maybe we should have a lean stdlib and a thicker standard distribution of selected packages <ul> <li>who would decide?</li> <li>same problem exists then of depending on it. How to remove stuff?</li> <li>Steve Dower <a href="">would rather see</a> a smaller standard library with some kind of "standard distribution" of PyPI modules that is curated by the core developers.</li> </ul></li> <li>A leaner stdlib could speed up Python version schedules and reduce burden on core devs to maintain seldom used packages.</li> </ul></li> <li>See? can of worms.</li> <li>In any case, all this would require a PEP, so we have to wait until we have a PEP process decided on.</li> </ul></li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Data Science portal for Home Assistant launched</strong></a></p> <ul> <li>via Paul Cutler</li> <li>Home Assistant is launching a data science portal to teach you how you can learn from your own smart home data. </li> <li>In 15 minutes you setup a local data science environment running reports.</li> <li>A core principle of Home Assistant is that a user has complete ownership of their personal data. A users data lives locally, typically on the SD card in their Raspberry Pi</li> <li>The Home Assistant Data Science website is your one-stop-shop for advice on getting started doing data science with your Home Assistant data.</li> <li>To accompany the website, we have created a brand new Add-on <a href="">JupyterLab lite</a>, which allows you to run a data science IDE called <a href="">JupyterLab</a> directly on your Raspberry Pi hosting Home Assistant. You do your data analysis locally, your data never leaves your local machine. </li> <li>When you build something cool, you can share the notebook without the results, so people can run it at their homes too.</li> <li>We have also created a Python library called the <a href="">HASS-Data-Detective</a> which makes it super easy to get started investigating your Home Assistant data using modern data science tools such as <a href="">Pandas</a>.</li> <li>Check out the <a href="">Getting Started notebook</a></li> <li>IoT aside: I finally found my first IoT project: Recording in progress button.</li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>What's the future of the pandas library?</strong></a></p> <ul> <li>Kevin Markham over at</li> <li>pandas is gearing up to move towards a 1.0 release. Currently rc-ing 0.24</li> <li>Plans are to get there “early 2019”.</li> <li>Some highlights <ul> <li>method chaining - encouraged by core team <ul> <li>to encourage further, more methods will support chaining</li> </ul></li> <li>Apache arrow likely to be part of pandas backend sometime after 1.0</li> <li>Extension arrays - allow you to create custom data types</li> <li>deprications <ul> <li><code>inplace</code> parameter. It doesn’t work with chaining, doesn’t actually prevent copies, and causes codebase complexity</li> <li><code>ix</code> accessor, use <code>loc</code> and <code>iloc</code> instead</li> <li><code>Panel</code> data structure. Use <code>MultiIndex</code> instead</li> <li><code>SparseDataFrame</code>. Just use a normal <code>DataFrame</code></li> <li>legacy python support</li> </ul></li> </ul></li> </ul> <pstrong>Michael #4:</strong> <a href=""><strong>PyOxidizer</strong></a></p> <ul> <li><code>PyOxidizer</code> is a collection of Rust crates that facilitate building libraries and binaries containing Python interpreters.</li> <li><code>PyOxidizer</code> is capable of producing a single file executable - with all dependencies statically linked and all resources (like <code>.pyc</code> files) embedded in the executable</li> <li>The <em>Oxidizer</em> part of the name comes from Rust: executables produced by <code>PyOxidizer</code> are compiled from Rust and Rust code is responsible for managing the embedded Python interpreter and all its operations.</li> <li><code>PyOxidizer</code> is similar in nature to <a href="">PyInstaller</a>, <a href="">Shiv</a>, and other tools in this space. What generally sets <code>PyOxidizer</code> apart is <ul> <li>Produced executables contain an embedded, statically-linked Python interpreter</li> <li>have no additional run-time dependency on the target system</li> <li>runs everything from memory (as opposed to e.g. extracting Python modules to a temporary directory and loading them from there).</li> </ul></li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>Working With Files in Python</strong></a></p> <ul> <li>by Vuyisile Ndlovu on RealPython</li> <li>Very comprehensive write up on working with files and directories</li> <li>Includes legacy and modern methods. <ul> <li>Pay attention to pathlib parts if you are using 3.4 plus</li> <li>Also great for “if you used to do x, here’s how to do it with pathlib”.</li> </ul></li> <li>Included: <ul> <li>Directory listings</li> <li>getting file attributes</li> <li>creating directories</li> <li>file name pattern matching</li> <li>traversing directories doing stuff with the files in there</li> <li>creating temp directories and files</li> <li>deleting, copying, moving, renaming</li> <li>archiving with zip and tar including reading those</li> <li>looping over files</li> </ul></li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>$ python == $ python3?</strong></a></p> <ul> <li>via <a href="">David Furphy</a></li> <li><a href="">Homebrew tried</a> this recently &amp; got "persuaded" to reverse. </li> <li>Also in recent discussion of edits to <a href="">PEP394</a>, <a href="">GvR said</a> absolutely not now, probably not ever.</li> <li>Guido van Rossum <ul> <li>RE: <code>python</code> doesn’t exist on macOS as a command: Did you mean python2 there? In my experience macOS comes with python installed (and invoking Python 2) but no python2 link (hard or soft). In any case I'm not sure how this strengthens your argument.</li> <li>I'm also still unhappy with any kind of endorsement of python pointing to python3. When a user gets bitten by this they should receive an apology from whoever changed that link, not a haughty "the PEP endorses this".</li> <li>Regardless of what macOS does I think I would be happier in a future where python doesn't exist and one always has to specify python2 or python3. Quite possibly there will be an age where Python 2, 3 and 4 all overlap, and EIBTI.</li> </ul></li> </ul> <pstrong>Extras:</strong></p> <p><strong>Michael: <a href="">A letter to the Python community in Africa</a></strong></p> <ul> <li>via Anthony Shaw</li> <li>Believe the broader international Python and Software community can learn a lot from what so many amazing people are doing across Africa.</li> <li>e.g. The attendance of PyCon NA was 50% male and 50% female.</li> </ul> <pstrong>Joke:</strong> via Luke Russell: A: “Knock Knock” B: “Who’s There" A: ……………………………………………………………………………………….“Java”</p> <p>Also: Java 4EVER video is amazing: <a href=""></a></p>
Jan 26, 2019
#113 Python Lands on the Windows 10 App Store
<p>Sponsored by <a href=""></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>Advent of Code 2018 Solutions</strong></a></p> <ul> <li>Michael Fogleman</li> <li>Even if you didn’t have time or energy to do the 2018 AoC, you can learn from other peoples solutions. Here’s one set written up in a nice blog post.</li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Python Lands on the Windows 10 App Store</strong></a></p> <ul> <li>Python Software Foundation recently released Python 3.7 as an app on the official Windows 10 app store. </li> <li>Python 3.7 is now available to install from the Microsoft Store, meaning you no longer need to manually download and install the app from the official Python website.</li> <li>there is one limitation. “Because of restrictions on Microsoft Store apps, Python scripts may not have full write access to shared locations such as TEMP and the registry. </li> <li>Discussed with Steve Dower over on <a href="">Talk Python 191</a></li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>How I Built A Python Web Framework And Became An Open Source Maintainer</strong></a></p> <ul> <li>Florimond Manca</li> <li><a href="">Bocadillo</a> - “A modern Python web framework filled with asynchronous salsa”</li> <li>”<strong>maintaining an open source project is a marathon, not a sprint</strong>.”</li> <li>Tips at the end of the article include tips for the following topics, including recommendations and tool choices: <ul> <li><strong>Project definition</strong></li> <li><strong>Marketing &amp; Communication</strong></li> <li><strong>Community</strong></li> <li><strong>Project management</strong></li> <li><strong>Code quality</strong></li> <li><strong>Documentation</strong></li> <li><strong>Versioning and releasing</strong></li> </ul></li> </ul> <pstrong>Michael #4: Python maintainability score via</strong> <a href=""><strong>Wily</strong></a></p> <ul> <li>via Anthony Shaw</li> <li>A Python application for tracking, reporting on timing and complexity in tests</li> <li>Easiest way to calculate it is with wily <a href=""></a> … the metrics are ‘maintainability.mi’ and ‘maintainability.rank’ for a numeric and the A-F scale. <ul> <li>Build an index: wily build src</li> <li>Inspect report: wily report file</li> <li>Graph: wily graph file metric</li> </ul></li> </ul> <pstrong>Brian #5:</strong> <strong>A couple fun awesome lists</strong></p> <ul> <li><a href="">Awesome Python Security resources</a> <ul> <li>Tools <ul> <li>web framework hardening, ex:</li> <li>multi tools</li> <li>static code analysis, ex: bandit</li> <li>vulnerabilities and security advisories</li> <li>cryptography</li> <li>app templates</li> </ul></li> <li>Education <ul> <li>lots of resources for learning</li> </ul></li> <li>Companies</li> </ul></li> <li><a href="">Awesome Flake8 Extensions</a> <ul> <li>clean code</li> <li>testing, including <ul> <li><a href="">flake8-pytest</a> - Enforces to use <code>pytest</code>-style assertions</li> <li><a href="">flake8-mock</a> - Provides checking mock non-existent methods</li> </ul></li> <li>security</li> <li>documentation</li> <li>enhancements</li> <li>copyrights</li> </ul></li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>fastlogging</strong></a></p> <ul> <li>via Robert Young</li> <li>A faster replacement of the standard logging module with a mostly compatible API.</li> <li>For a single log file it is ~5x faster and for rotating log file ~13x faster.</li> <li>It comes with the following features: <ul> <li>(colored, if <code>colorama</code> is installed) logging to console</li> <li>logging to file (maximum file size with rotating/history feature can be configured)</li> <li>old log files can be compressed (the compression algorithm can be configured)</li> <li>count same successive messages within a 30s time frame and log only once the message with the counted value.</li> <li>log domains</li> <li>log to different files</li> <li>writing to log files is done in (per file) background threads, if configured</li> <li>configure callback function for custom detection of same successive log messages</li> <li>configure callback function for custom message formatter</li> <li>configure callback function for custom log writer</li> </ul></li> </ul> <pxtras:</p> <ul> <li>Michael: <a href="">My webcast on async</a>, Jan 24, 11am PT</li> <li>Michael: <a href="">Watch your YAML</a>! </li> </ul> <p>Joke: <code>&gt;&gt;&gt; import antigravity</code></p>
Jan 18, 2019
#112 Don't use the greater than sign in programming
<p>Sponsored by <a href=""></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>nbgrader</strong></a></p> <ul> <li><a href=""><strong>nbgrader: A Tool for Creating and Grading Assignments in the Jupyter Notebook</strong></a> <ul> <li>The Journal of Open Source Education, paper accepted 6-Jan-2019</li> </ul></li> <li><a href="">nbgrader documentation, including a intro video</a></li> <li>From the JOSE article: <ul> <li>“nbgrader is a flexible tool for creating and grading assignments in the Jupyter Notebook (Kluyver et al., 2016). nbgrader allows instructors to create a single, master copy of an assignment, including tests and canonical solutions. From the master copy, a student version is generated without the solutions, thus obviating the need to maintain two separate versions. nbgrader also automatically grades submitted assignments by executing the notebooks and storing the results of the tests in a database. After auto-grading, instructors can manually grade free responses and provide partial credit using the formgrader Jupyter Notebook extension. Finally, instructors can use nbgrader to leave personalized feedback for each student’s submission, including comments as well as detailed error information.”</li> </ul></li> <li>CS teaching methods have come a long ways since I was turning in floppies and code printouts.</li> </ul> <pstrong>Michael #2: <a href="">profanity-check</a></strong></p> <ul> <li>A fast, robust Python library to check for offensive language in strings.</li> <li><code>profanity-check</code> uses a linear SVM model trained on 200k human-labeled samples of clean and profane text strings.</li> <li>Making <code>profanity-check</code> both robust and extremely performant</li> <li>Other libraries like <a href="">profanity-filter</a> use more sophisticated methods that are much more accurate but at the cost of performance. <ul> <li>profanity-filter runs in 13,000ms vs 24ms for profanity-check in a benchmark</li> </ul></li> <li>Two ways to use: <ul> <li><code>predict(text)</code> → 0 or 1 (1 = bad)</li> <li><code>predict_prob(text)</code> → [0, 1] confidence interval (1 = bad)</li> </ul></li> </ul> <pstrong>Brian #3</strong>: <a href=""><strong>An Introduction to Python Packages for Absolute Beginners</strong></a></p> <ul> <li>Ever tried to explain the difference between module and package? Between package-in-the-directory-with-init sense and package-you-can-distribute-and-install-with-pip sense? Here’s the article to read beforehand.</li> <li>Modules, packages, using packages, installing, importing, and more.</li> <li>And that’s not even getting into flit and poetry, etc. But it’s a good place to start for people new to Python.</li> </ul> <pstrong>Michael #4: Python Dependencies and IoC</strong></p> <ul> <li>via Joscha Götzer</li> <li><strong>Open-closed principle</strong> is at work with these and is super valuable to testing (one of the SOLID principles): <em>Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification</em>.</li> <li>There is a huge debate around why Python doesn’t need DI or Inversion of Control (IoC), and a quick stackoverflow search yields multiple results along the lines of “python is a scripting language and dynamic enough so that DI/IoC makes no sense”. However, especially in large projects it might reduce the cognitive load and decoupling of individual components</li> <li><a href=""><strong>Dependency Injector</strong></a><strong>:</strong> I couldn’t get this one to work on windows, as it needs to compile some C libraries and some Visual Studio tooling was missing that I couldn’t really install properly. The library looks quite promising though, but sort of static with heavy usage of containers and not necessarily pythonic.</li> <li><a href=""><strong>Injector</strong></a>: The library that above mentioned article talks about, a little Java-esque</li> <li><a href=""><strong>pinject</strong></a>: Has been unmaintained for about 5 years, and only recently got new attention from some open source people who try to port it to python3. A product under Google copyright, and looks quite nice despite the lack of python3 bindings. Probably the most feature-rich of the listed libraries.</li> <li><a href=""><strong>python-inject</strong></a><strong>:</strong> I discovered that one while writing this email, not really sure if it’s any good. Nice use of type annotations and testing features</li> <li><a href=""><strong>di-py</strong></a>: Only works up to python 3.4, so I’ve also never tried it (I’m one of those legacy python haters, I’m sure you can relate 😄).</li> <li><a href=""><strong>Serum</strong></a>: This one is a little too explicit to my mind. It makes heavy use of context managers (literally with Context(...): everywhere 😉) and I’m not immediately sure how to work with it. In this way, it is quite powerful though. Interesting use of class decorators.</li> <li>And now on to my favorite and a repeated recommendation of mine around the internet→ <a href=""><strong>Haps</strong></a>: This lesser-known, lightweight library is sort of the new kid on the block, and really simple to use. As some of the other libraries, it uses type annotations to determine the kind of object it is supposed to instantiate, and automatically discovers the required files in your project folder. Haps is very pythonic and fits into apps of any size, helping to ensure modularization as the only dependency of your modules will be one of the types provided by the library. <a href="">Pretty good example here</a>.</li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>A Gentle Introduction to Pandas</strong></a></p> <ul> <li>Really a gentle introduction to the Pandas data structures <code>Series</code> and <code>DataFrame</code>.</li> <li>Very gentle, with console examples.</li> <li>Create series objects: <ul> <li>from an array</li> <li>from an array, and change the indexing</li> <li>from a dictionaries</li> <li>from a scalar, cool. didn’t know you could do that</li> </ul></li> <li>Accessing elements in a series</li> <li>DataFrames <ul> <li>sorting, slicing</li> <li>selecting by label, position</li> <li>statistics on columns</li> <li>importing and exporting data</li> </ul></li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>Don't use the greater than sign in programming</strong></a></p> <ul> <li>One simple thing that comes up time and time again is the use of the greater than sign as part of a conditional while programming. Removing it cleans up code.</li> <li>Let's say that I want to check that something is between 5 and 10. </li> <li>There are many ways I can do this</li> </ul> <pre><code> x &gt; 5 and 10 &gt; x 5 &lt; x and 10 &gt; x x &gt; 5 and x &lt; 10 10 &lt; x and x &lt; 5 x &lt; 10 and x &gt; 5 x &lt; 10 and 5 &lt; x </code></pre> <ul> <li>Sorry, one of those is incorrect. Go ahead and find out which one</li> <li>If you remove the use of the greater than sign then only 2 options remain <ul> <li><code>x &lt; 10 and 5 &lt; x</code></li> <li><code>5 &lt; x and x &lt; 10</code></li> <li>The last is nice because x is <strong>literally between</strong> 5 and 10</li> </ul></li> <li>There is also a nice way of expressing that <em>"x is outside the limits of 5 and 10”</em> <ul> <li><code>x &lt; 5 or 10 &lt; x</code></li> <li>Again, this expresses it nicely because x is <strong>literally outside</strong> of 5 to 10.</li> </ul></li> <li>Interesting comment: What is cleaner or easier to read comes down to personal taste. But how to express "all numbers greater than 1" without '>'? <ul> <li>ans: <code>1 &lt; allNumbers</code></li> </ul></li> </ul> <pstrong>Extras</strong></p> <p>Michael</p> <ul> <li><a href="">Teaching Python podcast</a> by Kelly Paredes &amp; Sean Tibor</li> <li><a href="">Github private repos (now free)</a></li> <li><a href="">EuroPython 2019 announced</a></li> <li><a href="">South African AWS Data Center coming</a> (via William H.)</li> <li><a href="">Pandas is dropping legacy Python support</a> any day now</li> </ul> <p><strong>Joke</strong>: <a href="">Harry Potter Parser Tongue</a> via Nick Spirit</p>
Jan 11, 2019
#111 loguru: Python logging made simple
<p>Sponsored by <a href=""></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>loguru:</strong></a> <a href=""><strong>Python logging made (stupidly) simple</strong></a></p> <ul> <li>Finally, a logging interface that is just slightly more syntax than print to do mostly the right thing, and all that fancy stuff like log rotation is easy to figure out.</li> <li>i.e. a logging API that fits in my brain.</li> <li>bonus: README is a nice tour of features with examples.</li> <li>Features: <ul> <li>Ready to use out of the box without boilerplate</li> <li>No Handler, no Formatter, no Filter: one function to rule them all</li> <li>Easier file logging with rotation / retention / compression</li> <li>Modern string formatting using braces style</li> <li>Exceptions catching within threads or main</li> <li>Pretty logging with colors</li> <li>Asynchronous, Thread-safe, Multiprocess-safe</li> <li>Fully descriptive exceptions</li> <li>Structured logging as needed</li> <li>Lazy evaluation of expensive functions</li> <li>Customizable levels</li> <li>Better datetime handling</li> <li>Suitable for scripts and libraries</li> <li>Entirely compatible with standard logging</li> <li>Personalizable defaults through environment variables</li> <li>Convenient parser</li> <li>Exhaustive notifier</li> </ul></li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Python gets a new governance model</strong></a></p> <ul> <li>by Brett Canon</li> <li>July 2018, Guido steps down</li> <li>Python progress has basically been on hold since then</li> <li>ended up with <a href="">7 governance proposals</a></li> <li>Voting was open to all core developers as we couldn't come up with a reasonable criteria that we all agreed to as to what defined an "active" core dev</li> <li>And the winner is ... In the end PEP 8016, the steering council proposal, won.</li> <li>it was a decisive win against second place</li> <li>PEP 8016 is heavily modeled on the Django project's organization (to the point that the PEP had stuff copy-and-pasted from the original Django governance proposal). <ul> <li>What it establishes is a steering council of five people who are to determine how to run the Python project. Short of not being able to influence how the council itself is elected (which includes how the electorate is selected), the council has absolute power.</li> <li>result of the vote prevents us from ever having the Python <em>project</em> be leaderless again, it doesn't directly solve how to guide the <em>language's</em> design.</li> </ul></li> <li>What's next? The next step is we elect the council. It's looking like nominations will be from Monday, January 07 to Sunday, January 20 and voting from Monday, January 21 to Sunday, February 03</li> <li>A key point I hope people understand is that while we solved the issue of project management that stemmed from Guido's retirement, the council will need to be given some time to solve the other issue of how to manage the design of Python itself.</li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>Why you should be using pathlib</strong></a></p> <ul> <li>Tour of pathlib from Trey Hunner</li> <li>pathlib combines most of the commonly used file and directory operations from <code>os</code>, <code>os.path</code>, and <code>glob</code>.</li> <li>uses objects instead of strings</li> <li>as of Python 3.6, many parts of stdlib support pathlib</li> <li>since <code>pathlib.Path</code> methods return <code>Path</code> objects, chaining is possible</li> <li>convert back to strings if you really need to for pre-3.6 code</li> <li>Examples: <ul> <li>make a directory: <code>Path('src/__pypackages__').mkdir(parents=True, exist_ok=True)</code></li> <li>rename a file: <code>Path('.editorconfig').rename('src/.editorconfig')</code></li> <li>find some files: <code>top_level_csv_files = Path.cwd().glob('*.csv')</code></li> <li>recursively: <code>all_csv_files = Path.cwd().rglob('*.csv')</code></li> <li>read a file: <code>Path('some/file').read_text()</code></li> <li>write to a file: <code>Path('.editorconfig').write_text('# config goes here')</code></li> <li><code>with open(path, mode) as x</code> works with Path objects as of 3.6</li> </ul></li> <li>Follow up article by Trey: <a href="">No really, pathlib is great</a></li> </ul> <pstrong>Michael #4:</strong> <a href=""><strong>Altair</strong></a> <strong>and</strong> <a href=""><strong>Altair Recipes</strong></a></p> <ul> <li>via Antonio Piccolboni (he wrote altair_recipes)</li> <li>Altair: Declarative statistical visualization library for Python <ul> <li>Altair is developed by Jake Vanderplas and Brian Granger</li> <li>By statistical visualization they mean: <ul> <li>The data source is a DataFrame that consists of columns of different data types (quantitative, ordinal, nominal and date/time).</li> <li>The DataFrame is in a tidy format where the rows correspond to samples and the columns correspond to the observed variables.</li> <li>The data is mapped to the visual properties (position, color, size, shape, faceting, etc.) using the group-by data transformation.</li> </ul></li> <li>Nice example that I can get behind</li> </ul></li> </ul> <pre><code> # cars = some Pandas data frame alt.Chart(cars).mark_point().encode( x='Horsepower', y='Miles_per_Gallon', color='Origin', ) </code></pre> <ul> <li>altair_recipes <ul> <li>Altair allows generating a wide variety of statistical graphics in a concise language, but lacks, by design, pre-cooked and ready to eat statistical graphics, like the boxplot or the histogram. </li> <li>Examples: <a href=""></a> </li> <li>They take a few lines only in altair, but I think they deserve to be one-liners. altair_recipes provides that level on top of altair. The idea is not to provide a multitude of creative plots with fantasy names (the way seaborn does) but a solid collection of classics that everyone understands and cover most major use cases: the scatter plot, the boxplot, the histogram etc. </li> <li>Fully documented, highly consistent API (see next package), 90%+ test coverage, maintainability grade A, this is professional stuff if I may say so myself.</li> </ul></li> </ul> <pstrong>Brian #5:</strong> <strong>A couple fun pytest plugins</strong></p> <ul> <li><a href="">pytest-picked</a> <ul> <li>Using <code>git status</code>, this plugin allows you to: <ul> <li>Run only tests from modified test files</li> <li>Run tests from modified test files first, followed by all unmodified tests</li> </ul></li> <li>Kinda hard to overstate the usefulness of this plugin to anyone developing or debugging a test. Very, very cool.</li> </ul></li> <li><a href="">pytest-clarity</a> <ul> <li>Colorized left/right comparisons</li> <li>Early in development, but already helpful.</li> <li>I recommend running it with -qq if you don’t normally run with -v/--verbose since it overrides the verbosity currently.</li> </ul></li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>Secure 🔒 headers and cookies for Python web frameworks</strong></a></p> <ul> <li>Python package called Secure, which sets security headers and cookies (as a start) for Python web frameworks.</li> <li>I was listening to the Talk Python To Me episode “Flask goes 1.0” with Flask maintainer David Lord. At the end of the interview he was asked about notable PyPI packages and spoke about Flask-Talisman, a third-party package to set security headers in Flask. As a security professional, it was surprising and encouraging to hear the maintainer of the most popular Python web framework speak passionately about a security package. </li> <li>Had been recently experimenting with emerging Python web frameworks and realized there was a gap in security packages. That inspired Caleb to (humbly) see if it were possible to make a package to correct that and I started with Responder and then expanded to support more frameworks. </li> <li>The outcome was Secure with functions to support aiohttp, Bottle, CherryPy, Falcon, hug, Pyramid, Quart, Responder, Sanic, Starlette and Tornado (most of these, if not all have been featured on Talk Python) and can also be utilized by frameworks not officially supported. The goal is to be minimalistic, lightweight and be implemented in a way that does not disrupt an individual framework’s design. </li> <li>I have had some great feedback and suggestions from the developer and OWASP community, including some awesome discussions with the OWASP Secure Project and the Sanic core team. </li> <li>Added support for Flask and Django too.</li> <li>Secure Cookies is nice in the mix</li> </ul> <pstrong>Extras:</strong></p> <p><strong>Michael:</strong> <a href=""><strong>SQLite bug impacts thousands of apps, including all Chromium-based browsers</strong></a></p> <ul> <li>See <a href=""></a></li> <li>Since this bug is triggered by an SQL command, general CPython usage should not be affected, and long as you don’t run arbitrary SQL-commands provided by the outside.</li> <li>Seems to NOT be a problem in CPython: <a href=""></a></li> </ul> <p><strong>Michael: Follow up to our AI and healthcare conversation</strong></p> <ul> <li>via Bradley Hintze</li> <li>I found your discussion of deep learning in healthcare interesting, no doubt because that is my area. I am the data scientist for the National Oncology Program at the Veterans Health Administration. </li> <li>I work directly with clinicians and it is my strong opinion that AI cannot take the job from the MD. It will however make caring for patients much more efficient as AI takes care of the low hanging fruit, it you will.</li> <li>Healthcare, believe it or not, is a science and an art. This is why AI is never going to make doctors obsolete. It will, however, make doctors more efficient and demanded a more sophisticated doctor -- one that understands AI enough to not only trust it but, crucially, comprehend its limits.</li> </ul> <p><strong>Michael: Upgrade to Python 3.7.2</strong></p> <ul> <li>If you install via home brew, it’s time for <code>brew update &amp;&amp; brew upgrade</code></li> </ul> <p><strong>Michael: New course!</strong> </p> <ul> <li><a href="">Introduction to Ansible</a></li> </ul>
Jan 05, 2019
#110 Python Year in Review 2018 Edition
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>This episode originally aired on Talk Python at <a href=""></a>.</p> <p>It's been a fantastic year for Python. Literally, every year is better than the last with so much growth and excitement in the Python space. That's why I've asked two of my knowledgeable Python friends, Dan Bader and Brian Okken, to help pick the top 10 stories from the Python community for 2018.</p> <p><strong>Guests</strong></p> <ul> <li>Brian Okken <a href="">@brianokken</a></li> <li>Dan Bader <a href="">@dbader_org</a></li> </ul> <p><strong>10: Python 3.7:</strong> </p> <ul> <li><a href="">Cool New Features in Python 3.7</a></li> </ul> <p><strong>9: Changes in versioning patterns</strong></p> <ul> <li><a href="">ZeroVer</a>: 0-based Versioning</li> <li><a href="">Calendar Versioning</a></li> <li><a href="">Semantic Versioning</a> 2.0.0</li> </ul> <p><strong>8: Python is becoming the world’s most popular coding language</strong></p> <ul> <li><a href="">Economist article</a></li> </ul> <p><strong>7: 2018 was the year data science Pythonistas == web dev Pythonistas</strong></p> <ul> <li><a href="">Python Developers Survey Results</a></li> <li><a href="">Covered in depth on Talk Python 176</a></li> </ul> <p><strong>6: Black</strong></p> <ul> <li><a href="">Project</a></li> <li>Soundgarden : <a href="">“Black Hole Sun”</a></li> </ul> <p><strong>5: New PyPI launched!</strong></p> <ul> <li><a href="">Python Package Index</a></li> </ul> <p><strong>4: Rise of Python in the embedded world</strong></p> <ul> <li><a href="">Covered at Python Bytes</a></li> </ul> <p><strong>3: Legacy Python's days are fading?</strong></p> <ul> <li>Python 2.7 -- <a href="">bugfix or security before EOL</a>?</li> <li><a href="">Python 2 death clock</a></li> </ul> <p><strong>2: It's the end of innocence for PyPi</strong></p> <ul> <li><a href="">Twelve malicious Python libraries found and removed from PyPI</a></li> </ul> <p><strong>1: Guido stepped down as BDFL</strong></p> <ul> <li><a href="">python-committers: Transfer of power</a></li> <li><a href="">Proposals for new governance structure</a></li> </ul>
Dec 26, 2018
#109 CPython byte code explorer
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>Python Descriptors Are Magical Creatures</strong></a></p> <ul> <li>an excellent discussion of understanding <code>@property</code> and Python’s descriptor protocol.</li> <li>discussion includes getter, setter, and deleter methods you can override.</li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Data Science Survey 2018 JetBrains</strong></a></p> <ul> <li>JetBrains polled over 1,600 people involved in Data Science and based in the US, Europe, Japan, and China, in order to gain insight into how this industry sector is evolving</li> <li>Key Takeaways <ul> <li>Most people assume that <strong>Python will remain the primary programming language in the field for the next 5 years</strong>.</li> <li>Python is currently the <strong>most popular language among data scientists</strong>. </li> <li>Data Science <strong>professionals tend to use Keras and Tableau</strong>, while amateur data scientists are more likely to prefer Microsoft Azure ML. </li> </ul></li> <li>Most common activities among pros and amateurs: <ul> <li>Data processing</li> <li>Data visualization</li> </ul></li> <li>Main programming language for data analysis <ul> <li>Python 57%</li> <li>R 15%</li> <li>Julia 0%</li> </ul></li> <li>IDEs and Editors <ul> <li>Jupyter 43%</li> <li>PyCharm 38%</li> <li>RStudio 23%</li> <li>…</li> </ul></li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong></strong></a></p> <ul> <li><code></code> is a one file python library that extends <a href="">memoization</a> across runs using a cache file.</li> <li>memoization is an incredibly useful technique that many self taught or on the job taught developers don’t know about, because it’s not obvious.</li> <li>example:</li> </ul> <pre><code> import cache @cache.cache() def expensive_func(arg, kwarg=None): # Expensive stuff here return arg </code></pre> <ul> <li>The <code>@cache.cache()</code> function can take multiple arguments. <ul> <li><code>@cache.cache(timeout=20)</code> - Only caches the function for 20 seconds.</li> <li><code>@cache.cache(fname="my_cache.pkl")</code> - Saves cache to a custom filename (defaults to hidden file <code>.cache.pkl</code>)</li> <li><code>@cache.cache(key=cache.ARGS[KWARGS,NONE])</code> - Check against args, kwargs or neither of them when doing a cache lookup.</li> </ul></li> </ul> <pstrong>Michael #4:</strong> <a href=""><strong>Setting up the data science tools</strong></a></p> <ul> <li>part of a larger video series</li> <li>set up. Tools to keras ultimately</li> <li>Tools <ul> <li>anaconda</li> <li>tensorflow</li> <li>Jupyter</li> <li>Keras</li> </ul></li> <li>good for true beginners </li> <li>setup and activate a condo venv </li> <li>Start up a notebook and switch envs </li> <li>use conda, rather than pip</li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>chartify</strong></a></p> <ul> <li>“Python library that makes it easy for data scientists to create charts.”</li> <li>from the docs: <ul> <li>Consistent input data format: Spend less time transforming data to get your charts to work. All plotting functions use a consistent tidy input data format.</li> <li>Smart default styles: Create pretty charts with very little customization required.</li> <li>Simple API: We've attempted to make to the API as intuitive and easy to learn as possible.</li> <li>Flexibility: Chartify is built on top of <a href="">Bokeh</a>, so if you do need more control you can always fall back on Bokeh's API.</li> </ul></li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>CPython byte code explorer</strong></a></p> <ul> <li>JupyterLab extension to inspect Python Bytecode</li> <li>via <a href="">Anton Helm</a></li> <li>by Jeremy Tuloup</li> <li>You’ll see exactly what it’s about if you watch the GIF movie at the github repo.</li> <li>Can’t think of a better way to understand Python bytecode quickly than to play a little with this</li> <li>Comparing versions of CPython: If you have several versions of Python installed on your machine (let's say in different conda environments), you can use the extension to check how the bytecode might differ.</li> <li>Nice visualization of different performance aspects of while vs. for at the end</li> </ul> <pxtras:</p> <p>Brian: </p> <ul> <li><a href="">“How the Internet is made.”</a></li> </ul>
Dec 18, 2018
#108 Spilled data? Call the PyJanitor
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>pyjanitor</strong></a> <strong>- for cleaning data</strong></p> <ul> <li>originally a port of an R package called janitor, now much more.</li> <li>“pyjanitor’s etymology has a two-fold relationship to “cleanliness”. Firstly, it’s about extending Pandas with convenient data cleaning routines. Secondly, it’s about providing a cleaner, method-chaining, verb-based API for common pandas routines.”</li> <li>functionality: <ul> <li>Cleaning columns name (multi-indexes are possible!)</li> <li>Removing empty rows and columns</li> <li>Identifying duplicate entries</li> <li>Encoding columns as categorical</li> <li>Splitting your data into features and targets (for machine learning)</li> <li>Adding, removing, and renaming columns</li> <li>Coalesce multiple columns into a single column</li> <li>Convert excel date (serial format) into a Python datetime format</li> <li>Expand a single column that has delimited, categorical values into dummy-encoded variables</li> </ul></li> <li>This <code>pandas</code> code:</li> </ul> <pre><code> df = pd.DataFrame(...) # create a pandas DataFrame somehow. del df['column1'] # delete a column from the dataframe. df = df.dropna(subset=['column2', 'column3']) # drop rows that have empty values in column 2 and 3. df = df.rename({'column2': 'unicorns', 'column3': 'dragons'}) # rename column2 and column3 df['newcolumn'] = ['iterable', 'of', 'items'] # add a new column. - looks like this with pyjanitor: df = ( pd.DataFrame(...) .remove_columns(['column1']) .dropna(subset=['column2', 'column3']) .rename_column('column2', 'unicorns') .rename_column('column3', 'dragons') .add_column('newcolumn', ['iterable', 'of', 'items']) ) </code></pre> <pstrong>Michael #2:</strong> <a href=""><strong>What Does It Take To Be An Expert At Python?</strong></a></p> <ul> <li>Presentation at PyData 2017 by James Powell</li> <li>Covers Python Data Model (dunder methods)</li> <li>Covers uses of Metaclasses</li> <li>All done very smoothly as a series of demos</li> <li>Pretty long and in depth, 1.5+ hours</li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>Awesome Python Applications</strong></a></p> <ul> <li>pypi is a great place to find great packages you can use as examples for the packages you write. Where do you go for application examples? Well, now you can go to <a href="">Awesome Python Applications</a>.</li> <li>categories of applications included: internet, audio, video, graphics, games, productivity, organization, communication, education, science, CMS, ERP (enterprise resource planning), static site generators, and a whole slew of developer related applications.</li> <li>Mahmoud is happy to have help filling this out, so if you know of a great open source application written in Python, go ahead and contribute to this, or open an issue on this project.</li> </ul> <pstrong>Michael #4:</strong> <a href=""><strong>Django Core no more</strong></a></p> <ul> <li>Write up by <a href="">James Bennett</a></li> <li>If you’re not the sort of person who closely follows the internals of Django’s development, you might not know there’s <a href="">a draft proposal</a> to drastically change the project’s governance.</li> <li>What’s up: Django the open-source project is OK right now, but difficulty in recruiting and retaining enough active contributors.</li> <li>Some of the biggest open-source projects dodge this by having, effectively, corporate sponsorship of contributions.</li> <li>Django has become sort of a victim of its own success: the types of easy bugfixes and small features that often are the path to growing new committers have mostly been done already in Django.</li> <li>Not managed to bring in new committers at a sufficient rate to replace those who’ve become less active or even entirely inactive, and that’s not sustainable for much longer.</li> <li>Under-attracting women contributors too</li> <li>Governance: Some parallels to what the Python core devs are experiencing now. Project leads BDFLs stepped down.</li> <li>The proposal: what I’ve proposed is the dissolution of “Django core”, and the revocation of almost all commit bits <ul> <li>Seems extreme but they were working much more as a team with PRs, etc anyway.</li> <li>Breaks down the barrier to needing to be on the core team to suggest, change anything.</li> <li>Two roles would be formalized — Mergers and Releasers — who would, respectively, merge pull requests into Django, and package/publish releases. But rather than being all-powerful decision-makers, these would be bureaucratic roles</li> </ul></li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>wemake django template</strong></a></p> <ul> <li>a cookie-cutter template for serious django projects with lots of fun goodies</li> <li>“This project is used to scaffold a <code>django</code> project structure. Just like <code> startproject</code> but better.”</li> <li>features: <ul> <li>Always <code>up-to-date</code> with the help of <code>[@dependabot](</code></li> <li><code>poetry</code> for managing dependencies</li> <li><code>mypy</code> for optional static typing</li> <li><code>pytest</code> for unit testing</li> <li><code>flake8</code> and <code>wemake-python-styleguide</code> for linting</li> <li><code>pre-commit</code> hooks for consistent development</li> <li><code>docker</code> for development, testing, and production</li> <li><code>sphinx</code> for documentation</li> <li><code>Gitlab CI</code> with full <code>build</code>, <code>test</code>, and <code>deploy</code> pipeline configured by default</li> <li><code>Caddy</code> with <code>https</code> and <code>http/2</code> turned on by default</li> </ul></li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>Django Hunter</strong></a></p> <ul> <li>Tool designed to help identify incorrectly configured Django applications that are exposing sensitive information.</li> <li>Why? March 2018: 28,165 thousand django servers are exposed on the internet, many are showing secret API keys, database passwords, amazon AWS keys. </li> <li>Example:</li> <li>Some complained this inferred Django was insecure and said it wasn’t. Others thought “There is a reasonable argument to be made that DEBUG should default to False.”</li> <li>One beginner, Peter, chimes in: <ul> <li>I probably have one of them, among my early projects that are on heroku and public GitHub repos. </li> <li>I did accidentally expose my aws password this way and all hell broke loose.</li> <li>The problem is that as a beginner, it wasn't obvious to me how to separate development and production settings and keep production stuff out of my public repository.</li> </ul></li> </ul> <pstrong>Extras</strong>:</p> <p>Michael: Thanks for having me on your show Brian: <a href=""></a></p> <p>Brian: open source extra: For Christmas, I want a dragon…</p> <blockquote> <p><a href=""></a> — Changelog (@changelog) </p> </blockquote> <p>Michael: Why did the multithreaded chicken cross the road?</p> <ul> <li>road the side get to the other of to</li> <li>to get the side to road the of other</li> <li>the side of to the to road other get</li> <li>to of the road to side other the get</li> </ul>
Dec 11, 2018
#107 Restructuring and searching data, the Python way
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>glom: restructuring data, the Python way</strong></a></p> <ul> <li>glom is a new approach to working with data in Python, featuring: <ul> <li>Path-based access for nested structure <ul> <li><code>data\['a'\]['b']['c']</code> → <code>glom(data, 'a.b.c')</code></li> </ul></li> <li>Declarative data transformation using lightweight, Pythonic specifications <ul> <li><code>glom(target, spec, **kwargs)</code> with options such as <ul> <li>a default value if value not found</li> <li>allowed exceptions</li> </ul></li> </ul></li> <li>Readable, meaningful error messages: <ul> <li><code>PathAccessError: could not access 'c', part 2 of Path('a', 'b', 'c')</code> is better than</li> <li><code>TypeError: 'NoneType' object is not subscriptable</code></li> </ul></li> <li>Built-in data exploration and debugging features <ul> <li><code>glom.Inspect(``**a*``,</code> <code>***kw*``)</code></li> <li>The <code>[**Inspect**](</code> specifier type provides a way to get visibility into glom’s evaluation of a specification, enabling debugging of those tricky problems that may arise with unexpected data.</li> </ul></li> </ul></li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Scientific GUI apps with TraitsUI</strong></a></p> <ul> <li>via Franklin Ventura</li> <li>They support: PyQt, wxPython, PySide, PyQt5</li> <li>People should be aware of and when combined with <a href="">Chaco</a> (again from Enthought) the graphing and controlling capabilities really are amazing.</li> <li>Tutorial: <a href="">Writing a graphical application for scientific programming using TraitsUI 6.0</a></li> <li>Really simple UI / API for mapping object(s) to GUIs and back.</li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>Pampy: The Pattern Matching for Python you always dreamed of</strong></a></p> <ul> <li>“Pampy is pretty small (150 lines), reasonably fast, and often makes your code more readable and hence easier to reason about.”</li> <li>uses <code>_</code> as the missing info in a pattern</li> <li><p>simple <code>match</code> signature of <code>match(input, pattern, action)</code></p></li> <li><p>Examples</p> <ul> <li>nested lists and tuples</li> </ul></li> </ul> <pre><code> from pampy import match, _ x = [1, [2, 3], 4] match(x, [1, [_, 3], _], lambda a, b: [1, [a, 3], b]) # =&gt; [1, [2, 3], 4] - dicts: pet = { 'type': 'dog', 'details': { 'age': 3 } } match(pet, { 'details': { 'age': _ } }, lambda age: age) # =&gt; 3 match(pet, { _ : { 'age': _ } }, lambda a, b: (a, b)) # =&gt; ('details', 3) </code></pre> <pstrong>Michael #4:</strong> <a href=""><strong>Google AI better than doctors at detecting breast cancer</strong></a></p> <ul> <li>Google’s deep learning AI called LYNA able to correctly identify tumorous regions in lymph nodes 99 per cent of the time.</li> <li>We think of the impact of AI as killing 'low end' jobs [see <a href=" robot&amp;client=firefox-b-1-ab&amp;tbm=isch&amp;source=iu&amp;ictx=1&amp;fir=HY_0DIJRLAdG1M%253A%252CTopRdw4TesZqKM%252C_&amp;usg=AI4_-kRZ13zsoCnKopq53qUGwZV3ho31EA&amp;sa=X&amp;ved=2ahUKEwjmpeOb8_DeAhWUKH0KHcTwDaAQ9QEwAHoECAUQBA#imgrc=scl2Ob_gs0DKUM:">poster</a>], but these are "doctor" level positions.</li> <li>The presence or absence of these ‘nodal metastases’ influence a patient’s prognosis and treatment plan, so accurate and fast detection is important.</li> <li>In a second trial, six pathologists completed a diagnostic test with and without LYNA’s assistance. With LYNA’s help, the doctors found it ‘easier’ to detect small metastases, and on average the task took half as long.</li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>2018 Advent of Code</strong></a></p> <p>Another winter break activity people might enjoy is practicing with code challenges. AoC is a fun tradition.</p> <ul> <li>a calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like.</li> <li>don't need a computer science background to participate</li> <li>don’t need a fancy computer; every problem has a solution that completes in at most 15 seconds on ten-year-old hardware.</li> <li>There’s a leaderboard, so you can compete if you want. Or just have fun.</li> <li>Past years available, back to 2015.</li> <li>Some extra tools and info: <a href="">awesome-advent-of-code</a></li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>Red Hat Linux 8.0 Beta released, now (finally) updated to use Python 3.6 as default instead of 2.7</strong></a></p> <ul> <li>First of all, my favorite comment was a correction to the title: legacy python *</li> <li>“<strong>Python 3.6 is the default Python implementation in RHEL 8</strong>; limited support for Python 2.7 is provided. No version of Python is installed by default.“ <ul> <li>Red Hat Enterprise Linux 8 is distributed with Python 3.6. The package is not installed by default. To install Python 3.6, use the yum install python3 command.</li> <li>Python 2.7 is available in the python2 package. However, <strong>Python 2 will have a shorter life cycle and its aim is to facilitate smoother transition to Python 3 for customers</strong>.</li> <li>Neither the default python package nor the unversioned /usr/bin/python executable is distributed with RHEL 8. <strong>Customers are advised to use python3 or python2 directly.</strong> Alternatively, administrators can configure the unversioned python command using the alternatives command. </li> </ul></li> <li><strong>Python scripts must specify major version in hashbangs</strong> at RPM build time <ul> <li>In RHEL 8, executable Python scripts are expected to use hashbangs (shebangs) specifying explicitly at least the major Python version. </li> </ul></li> </ul> <pstrong>Extras:</strong></p> <p>Michael: We were <a href=""><strong>featured on TechMeme</strong></a> Long Ride Home podcast. Check out their <a href=""><strong>podcast here</strong></a>. Thank you to <strong>Brian McCullough</strong>, the host of the show. I just learned about their show through this exchange but can easily see myself listening from time to time. It’s like Python Bytes, but for the wider tech world and less developer focused but still solid tech foundations.</p> <p><strong>Brian:</strong> First story was about glom. I had heard of glom before, but got excited after interviewing Mahmoud for <a href="">T&amp;C 55</a>, where we discussed the difficulty in testing if you use glom or DSLs in general. A twitter exchange and GH issue followed the episode, with Anthony Shaw. At one point, Ant shared this great joke from Brenan Kellar:</p> <blockquote> <p>A QA engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 99999999999 beers. Orders a lizard. Orders -1 beers. Orders a ueicbksjdhd. </p> <p>First real customer walks in and asks where the bathroom is. The bar bursts into flames, killing everyone.</p> <p>— Brenan Keller (@brenankeller) <a href="">November 30, 2018</a></p> </blockquote>
Dec 07, 2018
#106 Fluent query APIs on Python collections
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <strong>Dependency Management through a DevOps Lens</strong></p> <ul> <li><a href="">Python Application Dependency Management in 2018</a> - Hynek</li> <li>An opinionated comparison of one use case and <a href="">pipenv</a>, <a href="">poetry</a>, <a href="">pip-tools</a></li> <li>“We have more ways to manage dependencies in Python applications than ever. But how do they fare in production? Unfortunately this topic turned out to be quite polarizing and was at the center of a lot of heated debates. This is my attempt at an opinionated review through a DevOps lens.”</li> <li>Best disclaimer in a blog article ever: <ul> <li>“<strong>DISCLAIMER:</strong> The following technical opinions are mine alone and if you use them as a weapon to attack <em>people</em> who try to improve the packaging situation you’re objectively a bad person. Please be nice.”</li> </ul></li> <li><strong>Requirements:</strong> Solution needs to meet the following features: <ol> <li>Allow me specify my immediate dependencies (e.g. Django),</li> <li>resolve the dependency tree and lock all of them with their versions and ideally <a href=""><strong>hashes</strong></a> (<a href=""><strong>more on hashes</strong></a>),</li> <li>integrate somehow with <a href="">tox</a> so I can run my tests,</li> <li>and finally allow me to install a project with all its locked dependencies into a virtual environment of my choosing.</li> </ol></li> <li>Seem like reasonable wishes. So far, none of the solutions work perfectly.</li> <li>A good example of pointing out tooling issues with his use case while being respectful of the people involved in creating other tools.</li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Plugins made simple with</strong></a> <a href=""><strong>pluginlib</strong></a></p> <ul> <li>makes creating plugins for Python very simple</li> <li>it relies on metaclasses, but the average programmer can easily get lost dealing with metaclasses</li> <li>Main Features: <ul> <li>Plugins are validated when they are loaded (instead of when they are used)</li> <li>Plugins can be loaded through different mechanisms (modules, filesystem paths, entry points)</li> <li>Multiple versions of the same plugin are supported (The newest one is used by default)</li> <li>Plugins can be blacklisted by type, name, or version</li> <li>Multiple plugin groups are supported so one program can use multiple sets of plugins that won't conflict</li> <li>Plugins support conditional loading (examples: os, version, installed software, etc)</li> <li>Once loaded, plugins can be accessed through dictionary or dot notation</li> </ul></li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>How to Test Your Django App with Selenium and pytest</strong></a></p> <ul> <li>Bob Belderbos</li> <li>“In this article I will show you how to test a Django app with pytest and Selenium. We will test our <a href=""> platform</a> comparing the logged out homepage vs the logged in dashboard. We will navigate the DOM matching elements and more.”</li> </ul> <pstrong>Michael #4: <a href="">Fluent collection APIs</a> (<code>flupy</code> and <code>asq</code>)</strong></p> <ul> <li>flupy implements a fluent interface for chaining multiple method calls as a single python expression. </li> <li>All flupy methods return generators and are evaluated lazily in depth-first order. </li> <li>This allows flupy expressions to transform arbitrary size data in extremely limited memory.</li> <li>Example:</li> </ul> <pre><code> pipeline = flu(count()).map(lambda x: x**2) \ .filter(lambda x: x % 517 == 0) \ .chunk(5) \ .take(3) for item in pipeline: print(item) </code></pre> <ul> <li>The CLI in particular has been great for our data science team. Not everyone is super comfortable with linux-fu so having a cross-platform way to leverage python knowledge on the shell has been an easy win.</li> <li>Also if you are LINQ inclined: <strong><a href=""></a></strong></li> <li><code>asq</code> is simple implementation of a LINQ-inspired API for Python which operates over Python iterables, including a parallel version implemented in terms of the Python standard library multiprocessing module.</li> </ul> <pre><code> # ASQ &gt;&gt;&gt; from asq import query &gt;&gt;&gt; words = ["zero", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten"] &gt;&gt;&gt; query(words).order_by(len).then_by().take(5).select(str.upper).to_list() ['ONE', 'SIX', 'TEN', 'TWO', 'FIVE'] </code></pre> <pstrong>Brian #5:</strong> <strong>Guido blogging again</strong></p> <ul> <li><a href="">What to do with your computer science career</a></li> <li>Answering “A question about whether to choose a 9-5 job or be an entrepreneur” <ul> <li>entrepreneurship isn’t for everyone</li> <li>working for someone else can be very rewarding </li> <li>shoot for “better than an entry-level web development job”</li> </ul></li> <li>And “A question about whether AI would make human software developers redundant (not about what I think of the field of AI as a career choice)” <ul> <li>AI is about automating tasks that can be boring</li> <li>Software Engineering is never boring.</li> </ul></li> </ul> <pstrong>Michael #6: <a href="">Web apps in pure Python apps with Anvil</a></strong></p> <ul> <li>Design with our visual designer</li> <li>Build with nothing but Python</li> <li>Publish Instant hosting in the cloud or on-site</li> <li>Paid product but has a free version</li> <li>Covered on <a href="">Talk Python 138</a></li> </ul> <pstrong>Extras</strong>:</p> <ul> <li>Second Printing (P2) of “<a href="">Python Testing with pytest</a>”</li> </ul>
Dec 01, 2018
#105 Colorizing and Restoring Old Images with Deep Learning
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>Colorizing and Restoring Old Images with Deep Learning</strong></a></p> <ul> <li>Text interview by Charlie Harrington of Jason Antic, developer of <a href="">DeOldify</a></li> <li>A whole bunch of machine learning buzzwords that I don’t understand in the slightest combine to make a really cool to to make B&amp;W photos look freaking amazing.</li> <li>“This is a deep learning based model. More specifically, what I've done is combined the following approaches: <ul> <li><strong>Self-Attention Generative Adversarial Network</strong></li> <li>Training structure inspired by (but not the same as) <strong>Progressive Growing of GANs.</strong></li> <li><strong>Two Time-Scale Update Rule.</strong></li> <li><strong>Generator Loss</strong> is two parts: One is a basic Perceptual Loss (or Feature Loss) based on VGG16. The second is the loss score from the critic.”</li> </ul></li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>PlatformIO IDE for VSCode</strong></a></p> <ul> <li>via Jason Pecor</li> <li>PlatformIO is an open source ecosystem for IoT development</li> <li>Cross-platform IDE and unified debugger. Remote unit testing and firmware updates</li> <li>Built on Visual Studio Code which has a nice extension for Python</li> <li>PlatformIO, combined with the features of VSCode provides some great improvements for project development over the standard Arduino IDE for Arduino-compatible microcontroller based solutions.</li> <li>Some of these features are paid, but it’s a reasonable price</li> <li>With Python becoming more popular for microcontroller design, as well, this might be a very nice option for designers.</li> <li>And for Jason’s, specifically, it provides a single environment that can eventually be configured to handle doing the embedded code design, associated Python supporting tools mods, and HDL development. </li> <li>The PlatformIO Core written in Python. Python 2.7 (hiss…)</li> <li>Jason’s test drive video from Tuesday: <a href=";"><strong>Test Driving PlatformIO IDE for VSCode</strong></a></li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>Python Data Visualization 2018: Why So Many Libraries?</strong></a></p> <ul> <li>Nice overview of visualization landscape, by Anaconda team</li> <li>Differentiating factors, API types, and emerging trends</li> <li>Related: <a href="">Drawing Data with Flask and matplotlib</a> <ul> <li>Finally! A really simple example app in Flask that shows how to both generate and display matplotlib plots.</li> <li>I was looking for something like this about a year ago and didn’t find it.</li> </ul></li> </ul> <pstrong>Michael #4:</strong> <a href=""><strong> - VS Code in the cloud</strong></a></p> <ul> <li>Full Visual Studio Code, but in your browser</li> <li>Code in the browser</li> <li>Access up to 96 cores</li> <li>VS Code + extensions, so all the languages and features</li> <li>Collaborate in real time, think google docs</li> <li>Access linux from any OS</li> <li>Note: They sponsored an episode of Talk Python To Me, but this is not an ad here...</li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>By Welcoming Women, Python’s Founder Overcomes Closed Minds In Open Source</strong></a></p> <ul> <li>Forbes’s article about Guido and the Python community actively working to get more women involved in core development as well as speaking at conferences.</li> <li>Good lessons for other projects, and work teams, about how you cannot just passively “let people join”, you need to work to make it happen.</li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>Machine Learning Basics</strong></a></p> <ul> <li>From Anna-Lena Popkes</li> <li>Plain python implementations of basic machine learning algorithms</li> <li>Repository contains implementations of basic machine learning algorithms in plain Python (modern Python, yay!)</li> <li>All algorithms are implemented from scratch without using additional machine learning libraries. </li> <li>Goal is to provide a basic understanding of the algorithms and their underlying structure, not to provide the most efficient implementations.</li> <li>Most of the algorithms <ul> <li><a href="">Linear Regression</a></li> <li><a href="">Logistic Regression</a></li> <li><a href="">Perceptron</a></li> <li><a href="">k-nearest-neighbor</a></li> <li><a href="">k-Means clustering</a></li> <li><a href="">Simple neural network with one hidden layer</a></li> <li><a href="">Multinomial Logistic Regression</a></li> <li><a href="">Decision tree for classification</a></li> <li><a href="">Decision tree for regression</a></li> </ul></li> <li>Anna-Lena was on Talk Python on 186:</li> </ul> <p>Extras:</p> <ul> <li>Michael: <a href=""><strong>PSF Fellow Nominations are open</strong></a></li> <li>Michael: <a href=""><strong>Shiboken has no meaning</strong></a></li> <li>Brian: <a href=""><strong>Python 3.7 runtime now available in AWS Lambda</strong></a></li> </ul>
Nov 23, 2018
#104 API Evolution the Right Way
<h1>Python Bytes 104</h1> <p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Michael #0.1: Chapters and play at</strong></p> <ul> <li>Chapters are now in the mp3 file</li> <li>Play at button on the website (doesn’t work on iOS unless you click the play to start it)</li> </ul> <p><strong>Michael #0.2: <a href="">Become a friend of the show</a></strong></p> <ul> <li><a href=""></a></li> <li>Or just click “friends of the show” in the navbar</li> </ul> <pstrong>Brian #1:</strong> <a href=""><strong>wily: A Python application for tracking, reporting on timing and complexity in tests and applications.</strong></a></p> <ul> <li>Anthony Shaw (aka “Friend of the Show”, aka “Ant”)</li> <li><em>(if listing 2 “aliases, do you just put one “aka” or one per alias?)</em></li> <li>I should cover this on Test &amp; Code for the content of the package. But it’s the actual packaging that I want to talk about today.</li> <li>Wily is a code base that can be used as an example of embracing <code>pyproject.toml</code> (pyproject.toml discussed on <a href="">PB 100</a> and <a href="">T&amp;C 52</a>)</li> <li>A real nice clean project using newer packaging tools that also has some frequently used bells and whistles</li> <li>NO file</li> <li>wily’s pyproject.toml includes <ul> <li>flit packaging, metadata, scripts </li> <li>tox configuration</li> <li>black configuration</li> </ul></li> <li>project also has <ul> <li>testing done on TravisCI</li> <li>rst based docs and readthedocs updates</li> <li>code coverage</li> <li>black pre-commit for wily</li> <li>pre-commit hook for your project to run wily</li> <li> that includes code of conduct</li> <li> with a nice format</li> <li>tests using pytest</li> </ul></li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>Latest VS Code has Juypter support</strong></a></p> <ul> <li>In this release, closed a total of 49 issues, including: <ul> <li>Jupyter support: import notebooks and run code cells in a Python Interactive window</li> <li>Use new virtual environments without having to restart Visual Studio Code</li> <li>Code completions in the debug console window</li> <li>Improved completions in language server, including recognition of namedtuple, and generic types</li> </ul></li> <li>The extension now contains new editor-centric interactive programming capabilities built on top of Jupyter.</li> <li>have Jupyter installed in your environment (e.g. set your environment to Anaconda) and type #%% into a Python file to define a Cell. You will notice a “Run Cell” code lens will appear above the #%% line:</li> <li>Cells in the Jupyter Notebook will be converted to cells in a Python file by adding #%% lines. You can run the cells to view the notebook output in Visual Studio code, including plots</li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>API Evolution the Right Way</strong></a></p> <ul> <li>A. Jesse Jiryu Davis</li> <li>adding features</li> <li>removing features</li> <li>adding parameters</li> <li>changing behavior</li> </ul> <pstrong>Michael #4:</strong> <a href=""><strong>PySimpleGUI now on Qt</strong></a></p> <ul> <li>Project by Mike B</li> <li>Covered back on <a href=""></a></li> <li>Simple declarative UI “builder”</li> <li>Looking to take your Python code from the world of command lines and into the convenience of a GUI?</li> <li>Have a Raspberry Pi with a touchscreen that's going to waste because you don't have the time to learn a GUI SDK?</li> <li>Look no further, you've found your GUI package.</li> <li>Now supports Qt</li> <li>Modern Python only</li> <li>More frameworks likely coming</li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>Comparison of the 7 governance PEPs</strong></a></p> <ul> <li>Started by <a href="">Victor Stinner</a></li> <li>The different PEPs are compared by: <ul> <li>hierarchy</li> <li>number of people involved</li> <li>requirements for candidates to be considered for certain positions</li> <li>elections: who votes, and how</li> <li>term limits</li> <li>no confidence vote</li> <li>teams/experts</li> <li>PEP process</li> <li>core dev promotion and ejection</li> <li>how governance will be updated</li> <li>code of conduct</li> </ul></li> <li><a href="">PEP 8000</a>, <strong>Python Language Governance Proposal Overview</strong>: <ul> <li><a href="">PEP 8010</a> - <strong>The Technical Leader Governance Model</strong></li> <li>continue status quo (ish)</li> <li><a href="">PEP 8011</a> - <strong>Python Governance Model Lead by Trio of Pythonistas</strong></li> <li>like status quo but with 3 co-leaders</li> <li><a href="">PEP 8012</a> - <strong>The Community Governance Model</strong></li> <li>no central authority</li> <li><a href="">PEP 8013</a> - <strong>The External Governance Model</strong></li> <li>non-core oversight</li> <li><a href="">PEP 8014</a> - <strong>The Commons Governance Model</strong></li> <li>core oversight</li> <li><a href="">PEP 8015</a> - <strong>Organization of the Python community</strong></li> <li>push most decision-making to teams</li> <li><a href="">PEP 8016</a> - <strong>The Steering Council Model</strong></li> <li>bootstrap iterating on governance</li> </ul></li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>Shiboken</strong></a> (from Qt for Python project)</p> <ul> <li>From PySide2 (AKA Qt for Python) project</li> <li>Generate Python bindings from arbitrary C/C++ code</li> <li>Has a Typesystem (based on XML) which allows modifying the obtained information to properly represent and manipulate the C++ classes into the Python World.</li> <li>Can remove and add methods to certain classes, and even modify the arguments of each function, which is really necessary when both C++ and Python collide and a decision needs to be made to properly handle the data structures or types.</li> <li><a href="">Qt for Python: under the hood</a></li> <li><a href="">Write your own Python bindings</a></li> <li>Other options include: <ul> <li>CFFI (example <a href=""></a>)</li> <li>Cython (example: <a href="">via shamir.stav</a>)</li> </ul></li> </ul> <pstrong>Extras:</strong> </p> <ul> <li>Michael: <a href=""><strong>Mission Python: Code a Space Adventure Game! book</strong></a></li> <li>Michael: <a href=""><strong>PyCon tickets are on sale</strong></a></li> <li>Michael: <a href=""><strong>PyCascade tickets are on sale</strong></a></li> </ul>
Nov 17, 2018
#103 Getting to 10x (results for developers)
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <pstrong>Brian #1:</strong> <a href=""><strong>FEniCS</strong></a></p> <ul> <li>“FEniCS is a popular open-source (<a href="">LGPLv3</a>) computing platform for solving partial differential equations (PDEs). FEniCS enables users to quickly translate scientific models into efficient finite element code. With the high-level Python and C++ interfaces to FEniCS, it is easy to get started, but FEniCS offers also powerful capabilities for more experienced programmers. FEniCS runs on a multitude of platforms ranging from laptops to high-performance clusters.”</li> <li>Solves partial differential equations efficiently with a combination of C++ and Python.</li> <li>Can be run on a desktop/laptop or deployed to a supercomputer with thousands of parallel processes.</li> <li>is a <a href="">NumFOCUS</a> fiscally supported project</li> <li>“makes the implementation of the mathematical formulation of a system of partial differential equations almost seamless.” - Sébastien Brisard</li> <li>“FEniCS is in fact a C++ project with a full-featured Python interface. The library itself generates C++ code on-the-fly, that can be called (on-the-fly) from python. It's almost magical... Under the hood, it used to use SWIG, and recently moved to pybind11. I guess the architecture that was set up to achieve this level of automation might be useful in other situations.” - Sébastien Brisard</li> </ul> <pstrong>Michael #2:</strong> <a href=""><strong>cursive_re</strong></a></p> <ul> <li>via <a href="">Christopher Patti</a>, created by Bogdan Popa</li> <li>Readable regular expressions for Python 3.6 and up.</li> <li>It’s a tiny Python library made up of combinators that help you write regular expressions you can read and modify six months down the line.</li> <li>Best understood via an example:</li> </ul> <pre><code> &gt;&gt;&gt; hash = text('#') &gt;&gt;&gt; hexdigit = any_of(in_range('0', '9') + in_range('a', 'f') + in_range('A', 'F')) &gt;&gt;&gt; hexcolor = ( ... beginning_of_line() + hash + ... group(repeated(hexdigit, exactly=6) | repeated(hexdigit, exactly=3)) + ... end_of_line() ... ) &gt;&gt;&gt; str(hexcolor) '^\\#([a-f0-9]{6}|[a-f0-9]{3})$' </code></pre> <ul> <li>Has automatic escaping for <code>[</code> and <code>\</code> etc: <code>str(any_of(text("[]"))) → '[\\[\\]]'</code></li> <li>Easily testable / inspectable. Just call <code>str</code> on any expression.</li> </ul> <pstrong>Brian #3:</strong> <a href=""><strong>pyimagesearch</strong></a></p> <ul> <li>Adrian Rosebrock is focused on teaching OpenCV with Python</li> <li>Just a really cool resource of integrating computer vision and Python. Both free and paid resources.</li> <li>He had one of the most successful tech learning kickstarters (ever?) on this topic: <a href=""></a></li> </ul> <pstrong>Michael #4:</strong> <a href=""><strong>Visualization of Python development up till 2012</strong></a></p> <ul> <li>via <a href="">Ophion Group (on twitter)</a></li> <li>mercurial (hg) source code repository commit history </li> <li>August 1990 - June 2012 (cpython 3.3.0 alpha)</li> <li>Watch the first minute, then click ahead minute at a time and watch for a few seconds to get the full feel </li> <li>Really interesting to see a visual representation of the growth of an open source ecosystem</li> <li>Built with Gource: <a href=""></a> <ul> <li>Amazing video of the history gource and its visualization of various projects: <a href=""></a></li> </ul></li> <li>Who wants to build this for 2012-present?</li> <li>Would make an amazing lightning talk!</li> </ul> <pstrong>Brian #5:</strong> <a href=""><strong>Getting to 10x (Results): What Any Developer Can Learn from the Best</strong></a></p> <ul> <li>Forget the “10x” bit if that term is fighting words. - Brian’s advice <ul> <li>How about just “<strong>ways to improve your effectiveness as a developer</strong>”?</li> </ul></li> <li>“… there is a clear path to excellence. People aren’t born great developers. They get there through focused, deliberate practice.”</li> <li>traits of great developers <ul> <li>problem solver</li> <li>skilled</li> <li>mentor/teacher</li> <li>excellent learner</li> <li>passionate</li> </ul></li> <li>traits to avoid: <ul> <li>incompetent</li> <li>arrogant</li> <li>uncooperative</li> <li>unmotivated</li> <li>stubborn</li> </ul></li> <li>Focus on your strengths more than your weaknesses</li> <li>Pick 1 thing to improve on this week and focus on it relentlessly</li> </ul> <pstrong>Michael #6:</strong> <a href=""><strong>Chaos Toolkit</strong></a></p> <ul> <li>Chaos Engineering is the discipline of experimenting on a distributed system in order to build confidence in the system's capability to withstand turbulent conditions in production.</li> <li>Netflix uses the chaos monkey (et. al.) on their systems. Covered on <a href=""></a> </li> <li>The Chaos Toolkit aims to be the simplest and easiest way</li> <li>to explore building, and automating, your own Chaos Engineering Experiments.</li> <li>Integrates with Kubernetes, AWS, Google Cloud, Microsoft Azure, etc.</li> <li>To give you an idea, here are some things it can do to aws: <ul> <li>lambda: <code>delete_function_concurrency</code> Removes concurrency limit applied to the specified Lambda</li> <li><code>stop_instance</code> Stop a single EC2 instance. You may provide an instance id explicitly or, if you only specify the AZ, a random instance will be selected.</li> </ul></li> </ul> <pstrong>Extras:</strong> </p> <ul> <li>MK: <a href="">Malicious Python Libraries Found &amp; Removed From PyPI</a></li> <li>MK: <a href="">Some really long type names</a></li> <li>Brian: <a href="">Deep dive into pyproject.toml and the future of Python packaging with Brett Cannon</a> <ul> <li>follow up from <a href="">episode 100 Python Bytes</a></li> </ul></li> </ul>
Nov 08, 2018
#102 Structure of a Flask Project
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>QuantEcon</strong></a></p> <ul> <li>“Open source code for economic modeling”</li> <li>“QuantEcon is a <a href="">NumFOCUS</a> fiscally sponsored project dedicated to development and documentation of modern open source computational tools for economics, econometrics, and decision making.”</li> <li>Educational resource that includes: <ul> <li>Lectures, workshops, and seminars</li> <li>Cheatsheets for scientific programming in Python and Julia</li> <li>Notebooks</li> <li> : open source Python code library for economics</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Structure of a Flask Project</strong></a></p> <ul> <li>Flask is very flexible, it has no certain pattern of a project folder structure. Here are some suggestions.</li> <li>I always keep this one certain rule when writing modules and packages: <ul> <li>“Don't backward import from root <code></code>.”</li> </ul></li> <li>Candidate structure:</li> </ul> <pre><code> project/ models/ ... routes/ ... templates/ base.html post.html ... services/ </code></pre> <ul> <li>Love it! To this, I would rename <code>routes</code> to <code>views</code> or <code>controllers</code> and add a <code>viewmodels</code> folder and viewmodels themselves.</li> <li>Brian, see anything missing? <ul> <li>ya. tests. :)</li> </ul></li> <li>Another famous folder structure is app based structure, which means things are grouped bp application</li> <li>I (Michael) STRONGLY recommend <a href=""><strong>Flask blueprints</strong></a></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Overusing lambda expressions in Python</strong></a> <a href=""></a></p> <ul> <li>lambda expressions vs defined functions <ol> <li>They can be immediately passed around (no variable needed)</li> <li>They can only have a single line of code within them</li> <li>They return automatically</li> <li>They can’t have a docstring and they don’t have a name</li> <li>They use a different and unfamiliar syntax</li> </ol></li> <li>misuses: <ul> <li>naming them. Just write a function instead</li> <li>calling a single function with a single argument : just use that func instead</li> </ul></li> <li>overuse: <ul> <li>if they get complex, even a little bit, they are hard to read</li> <li>has to be all on one line, which reduces readibility</li> <li>map and filter : use comprehensions instead</li> <li>using custom lambdas instead of using operators from the <a href="">operator module</a>. </li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Asyncio in Python 3.7</strong></a></p> <ul> <li>by Cris Medina</li> <li>The release of Python 3.7 introduced a number of changes into the async world. <ul> <li>Some may even affect you even if you don’t use asyncio.</li> </ul></li> <li>New Reserved Keywords: The async and await keywords are now reserved. <ul> <li>There’s already quite a few modules broken because of this. However, the fix is easy: rename any variables and parameters.</li> </ul></li> <li>Context Variables: Version 3.7 now allows the use of context variables within async tasks. If this is a new concept to you, it might be easier to picture it as global variables whose values are local to the currently running coroutines.</li> <li>Python has similar constructs for doing this very thing across threads. However, those were not sufficient in async-world</li> <li>New <code></code> function <ul> <li>With a call to <code></code>, we can now automatically create a loop, run a task on it, and close it when complete.</li> </ul></li> <li>Simpler Task Management: Along the same lines, there’s a new asyncio.create_task() function that helps make tasks that inside the current loop, instead of having to get the loop first and calling create task on top of it.</li> <li>Simpler Event Loop Management: The addition of asyncio.get_running_loop() will help determine the active event loop, and catch a RuntimeError if there’s no loop running. </li> <li>Async Context Managers: Another quality-of-life improvement. We now have the asynccontextmanager() decorator for producing async context managers without the need for a class that implements __aenter__() or __aexit__(). </li> <li>Performance Improvements: Several functions are now optimized for speed, some were even reimplemented in C. Here’s the list: <ul> <li>asyncio.get_event_loop() is now 15 times faster.</li> <li>asyncio.gather() is 15% faster.</li> <li>asyncio.sleep() is two times faster when the delay is zero or negative.</li> <li>asyncio.Future callback management is optimized.</li> <li>Reduced overhead for asyncio debug mode.</li> </ul></li> <li>Lots lots more</li> </ul> <p><strong>Brian #5: Giving thanks with</strong> <code>**pip thank**</code></p> <ul> <li>proposal: <strong><a href=""></a></strong></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Getting Started With Testing in Python</strong></a></p> <ul> <li>by Anthony Shaw, 33 minutes reading time according to Instapaper</li> <li>Automated vs. Manual Testing</li> <li>Unit Tests vs. Integration Tests: A unit test is a smaller test, one that checks that a single component operates in the right way. A unit test helps you to isolate what is broken in your application and fix it faster.</li> <li>Compares unittest, nose or nose2, pytest</li> <li>Covers things like: <ul> <li>Writing Your First Test</li> <li>Where to Write the Test</li> <li>How to Structure a Simple Test</li> <li>How to Write Assertions</li> <li>Dangers of Side Effects</li> </ul></li> <li>Testing in PyCharm and VS Code</li> <li>Testing for Web Frameworks Like Django and Flask</li> <li>Advanced Testing Scenarios</li> <li>Even: Testing for Security Flaws in Your Application</li> </ul> <p><strong>Extras:</strong> </p> <ul> <li><strong>MK:</strong> <a href=""><strong>Hack ur name</strong></a> — aka Pivot me bro (done in Python: <a href=""></a> ) by Vicki Boykis</li> <li><strong>MK</strong>: <a href=""><strong>Python 3.7.1 and 3.6.7 Are Now Available</strong></a></li> <li><strong>MK</strong>: Click-Driven Development (CDD) - via @<a href="">tombaker</a> <ul> <li>Use Python Click package to mock up suite of commands w/options/args. </li> <li>Decorated functions print description of intended results. </li> <li>Replace placeholders with code.</li> </ul></li> </ul>
Oct 31, 2018
#101 Nobel Prize awarded to a Python convert
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Asterisks in Python: what they are and how to use them</strong></a></p> <ul> <li>I just ** love *s</li> <li>Using <code>*</code> and <code>**</code> to pass arguments to a function <ul> <li><code>*</code> for list, <code>**</code> for keyword arguments from a dictionary</li> </ul></li> <li>Using <code>*</code> and <code>**</code> to capture arguments passed into a function</li> <li>Using <code>*</code> to accept keyword-only arguments</li> <li>Using <code>*</code> to capture items during tuple unpacking <ul> <li>you can capture the rest if you only want to grab a few</li> </ul></li> <li>Using <code>*</code> to unpack iterables into a list/tuple</li> <li>Using <code>**</code> to unpack dictionaries into other dictionaries</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>responder web framework</strong></a></p> <ul> <li>From Kenneth Reitz — A familiar HTTP Service Framework</li> <li>Already has 1,393 github stars</li> <li>Flask-like but with async support and <ul> <li>A pleasant API, with a single import statement.</li> <li>Class-based views without inheritance.</li> <li>ASGI framework, the future of Python web services.</li> <li>WebSocket support!</li> <li>The ability to mount any ASGI / WSGI app at a subroute.</li> <li>f-string syntax route declaration.</li> <li>Mutable response object, passed into each view. No need to return anything.</li> <li>Background tasks, spawned off in a ThreadPoolExecutor.</li> <li>GraphQL (with GraphiQL) support!</li> <li>OpenAPI schema generation.</li> <li>Single-page webapp support</li> </ul></li> <li>Responder gives you the ability to mount another ASGI / WSGI app at a subroute</li> <li>uvicorn: powers responder and is built on top of uvloop</li> <li>asgi: <a href=""></a></li> </ul> <p><strong>Brian #3: Python Example resource:</strong> <a href=""><strong></strong></a></p> <ul> <li>Lots of examples</li> <li>Python basics including date time, strings, dictionaries </li> <li>pandas, matplotlib, tensorflow basics</li> <li>data structures and algorithms</li> <li>Nice reference, especially for people getting into Python for data science or scientific work.</li> </ul> <p><strong>Michael #4</strong>: <a href=""><strong>This year’s Nobel Prize in economics was awarded to a Python convert</strong></a></p> <ul> <li>Nordhaus and Romer “have designed methods that address some of our time’s most fundamental and pressing issues: long-term sustainable growth in the global economy and the welfare of the world’s population,”</li> <li>Notably for a 62-year-old economist of his distinction, he is a user of the <a href="">programming language Python</a>.</li> <li>Romer believes in making <a href="">research transparent</a>. He argues that openness and clarity about methodology is important for scientific research to gain trust.</li> <li>He tried to use Mathematica to share one of his studies in a way that anyone could explore every detail of his data and methods. It didn’t work. He says that Mathematica’s owner, Wolfram Research, made it too difficult to share his work in a way that didn’t require other people to use the proprietary software, too.</li> <li>Romer believes that open-source notebooks are the way forward for sharing research. He believes they support integrity, while proprietary software encourage secrecy. <strong>“The more I learn about proprietary software, the more I worry that objective truth might perish from the earth,”</strong> he wrote.</li> <li>Michael covered a similar story for the Nobel Prize in <a href="">Physics at CERN on Talk Python</a></li> <li>Jake Vanderplas Keynote at PyCon 2017: “The unexpected effectiveness of Python in Science”</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>More in depth TensorFlow</strong></a></p> <p><strong>Michael #6:</strong> <a href=""><strong>MAKERphone - an educational DIY mobile phone</strong></a></p> <ul> <li>MAKERphone is an educational DIY mobile phone designed to bring electronics and programming to the crowd in a fun and interesting way. <ul> <li>A <strong>fully functional</strong> mobile phone that you can code yourself</li> <li><strong>Games</strong> such as space invaders, pong, or snake</li> <li><strong>Apps</strong> such as a custom media player that only plays cat videos</li> <li>Programs in <strong>Arduino</strong> </li> <li>Lines of code in <strong>Python</strong> </li> <li>Your first working piece of code in <strong>Scratch</strong> </li> <li>A <strong>custom</strong> case </li> </ul></li> </ul> <p><strong>Extras:</strong> </p> <ul> <li>MK: <a href="">Around 62% of all Internet sites will run an unsupported PHP version in 10 weeks</a> <ul> <li>The highly popular PHP 5.x branch will stop receiving security updates at the end of the year.</li> </ul></li> </ul>
Oct 24, 2018
#100 The big 100 with special guests
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>Special guests:</p> <ul> <li><strong><a href="">Anthony Shaw</a></strong></li> <li><a href=""><strong>Dan Bader</strong></a></li> <li><strong><a href="">Brett Cannon</a></strong></li> <li><strong><a href="">Nina Zakharenko</a></strong></li> </ul> <p><strong>Brian #1:</strong> <a href=""><strong>poetry</strong></a></p> <ul> <li>“poetry is a tool to handle dependency installation as well as building and packaging of Python packages. It only needs one file to do all of that: the new, standardized pyproject.toml. In other words, poetry uses pyproject.toml to replace, requirements.txt, setup.cfg, and the newly added Pipfile.”</li> <li>poetry <ul> <li>can be used for both application and library development</li> <li>handles dependencies and lock files </li> <li>strongly encourages virtual environment use (need specifically turn it off)</li> <li>can be used within an existing venv or be used to create a new venv</li> <li>automates package build process</li> <li>automates deployment to PyPI or to another repository</li> <li>CLI and the use model is very different than pipenv. Even if they produced the same files (which they don’t), you’d still want to try both to see which workflow works best for you. For me, I think poetry matches my way of working a bit more than pipenv, but I’m still in the early stages of using either.</li> </ul></li> <li>From <a href=""><strong>Python's New Package Landscape</strong></a> <ul> <li>“<a href="">PEP 517</a> and <a href="">PEP 518</a>—accepted in September 2017 and May 2016, respectively—changed this status quo by enabling package authors to select different build systems. Said differently, <strong>for the first time in Python, developers may opt to use a distribution build tool other than</strong> <code>**distutils**</code> <strong>or</strong> <code>**setuptools**</code><strong>. The ubiquitous</strong> <code>****</code> <strong>file is no longer mandatory in Python libraries.”</strong></li> </ul></li> <li><a href="">PEP 517 -- A build-system independent format for source trees</a></li> <li><a href="">PEP 518 -- Specifying Minimum Build System Requirements for Python Projects</a></li> <li>Another project that utilizes pyproject.toml is <a href="">flit</a>, which seems to overlap quite a bit with poetry, but I don’t think it does the venv, dependency management, dependency updating, etc.</li> <li>See also: <ul> <li><a href="">Clarifying PEP 518 (a.k.a. pyproject.toml)</a> - From Brett</li> </ul></li> <li>Question for @Brett C 517 and 518 still say “provisional” and not “final”. What’s that mean? <ul> <li>We are still allowed to tweak it as necessary before it</li> </ul></li> <li>Biggest difference is poetry uses pyproject.toml (<a href="">PEP518</a>) instead of Pipfile. Replaces all others (, setup.cfg, requirements*.txt, manifest.IN) <ul> <li>Even its lock file is in TOML</li> </ul></li> <li>Author “does not like” pipenv, or some of the decisions it has made. Note that Kenneth has recently made some calls to introduce more discussion and openness with a PEP-style process called PEEP (<a href="">PipEnv Enhancement Proposals</a>). <ul> <li>E.g. uses a more extensive dependency resolver</li> </ul></li> <li>Pipenv does not support multiple environments (by design) making it useless for library development. Poetry makes this more open. See <a href=""></a></li> <li>Wait. Why am I doing your notes for you @Brian O ! (awesome. Thanks Ant.)</li> <li>Brett has had initial discussions on Twitter with both pipenv and poetry about possibly standardizing on a lockfile format so that’s the artifact these tools produce and everything else is tool preference</li> </ul> <p><strong>Anthony Shaw #2:</strong> <a href=""><strong>pylama</strong></a> <strong>and</strong> <a href=""><strong>radon</strong></a></p> <ul> <li>Have been investigating tools for measuring complexity and performance of code and how that relates to test</li> <li>If you can refactor your code so the tests still pass, the customers are still happy AND it’s simpler then that’s a good thing - right?</li> <li>Radon is a Python tool that leverages the AST to give statistics on Cyclomatic Complexity (number of decisions — nested if’s are bad), maintainability index (LoC &amp; Halstead) and Halstead (number of operations an complexity of AST).</li> <li>Radon works by adding a ComplexityVisitor to the AST.</li> <li>Another option is Ned Batchelder’s McCabe tool which measures the number of possible branches (similar to cyclomatic)</li> <li>All of these tools are combined in pylama - a code linter for Python and Javascript. Embeds pycodestyle, mccabe, radon, gjslint and pyflakes. </li> <li>Final goal is to have a pytest plugin that fails tests if you make your code more complicated</li> </ul> <p><strong>Nina Zakharenko #3:</strong> <strong>Tools for teaching Python</strong></p> <ul> <li>Teaching Python can come with hurdles — virtual environments, installing python3, pip, working with the command line. <ul> <li>Put out a call on twitter asking - “What software and tools do you use to teach Python?”.</li> <li>50 Responses, 414 votes, learned about lots of new tools. Read the <a href="">thread</a>. <ul> <li>27% use python or ipython repl</li> <li>13% use built-in IDLE</li> <li>39% use an IDE or editor - <a href="">Visual Studio Code</a>, <a href="">PyCharm</a>, <a href="">Atom</a>.</li> <li>21% use other (mix of local and <a href="">hosted Jupyter notebooks</a> and other responses)</li> </ul></li> </ul></li> <li>New tools I learned about: <ul> <li><a href="">Mu editor</a> - simple python editor, great for those completely new to programming. <ul> <li>Large buttons with common actions above the editor.</li> <li>Support for educational platforms <ul> <li>Integrates with hardware platforms -- adafruit <a href="">Circuit Playground,</a> <a href="">micro:bit</a></li> <li>PyGame</li> </ul></li> <li><a href="">Awesome tutorials</a></li> </ul></li> <li><a href=";WT.mc_id=pythonbytes-podcast-ninaz">Neuron plugin for VS Code</a>, <a href="">Hydrogen plugin for Atom</a> <ul> <li>Interactive coding environment, brings a taste of Jupyter notebooks into your editor.</li> <li>Targeted towards data scientists.</li> <li>Show evaluated values, output pane to display charts and graphs</li> <li>Import to/from Jupyter notebooks</li> </ul></li> <li><a href=""></a> - open source hosted cloud repl with reasonable free tier <ul> <li>project goal - <a href="">zero effort setup</a></li> <li>3 vertical panes: files, editor, repl, and a button to run the current code.</li> <li>no login, no signup needed to get started</li> <li>visual package installation - no running pip, requirements.txt automatically generated</li> <li>includes a debugger</li> </ul></li> <li><a href="">bpython</a> - Used it years ago, still an active project. <ul> <li>Fancy curses interface to the Python interactive interpreter. Windows, type hints, expected parameters lists.</li> <li>Really cool feature — you can rewind your session! Pops the last line, and the entire session is reevaluated. </li> <li>Easily reload imported modules. </li> </ul></li> </ul></li> <li>Honorable mentions: <ul> <li><a href="">Edublocks</a> - Teaching tool for kids, visually drag and drop blocks of Python code. <a href="">Open source</a>, created by <a href="">Joshua Lowe</a>, a brilliant 14 year old maker and programmer.</li> <li><a href="">pythonanywhere</a>, <a href=""></a>, <a href="">codesters</a>.</li> </ul></li> </ul> <p><strong>Dan Bader #4: My favorite tool of 2018:</strong> <a href=""><strong>“Black” code formatter</strong></a> by Łukasz Langa</p> <ul> <li>Black is the “uncompromising Python code formatter”</li> <li>An opinionated auto-formatter for your code (like YAPF/autopep for Python, or gofmt for golang who popularized the idea)</li> <li>Heard about it <a href="">in episode #73 by Brian</a> </li> <li>Started using it for some small tools, then rolled it out to the whole <a href=""></a> code base including our public example code repo (<a href=""></a>)</li> <li>Benefits are: <ul> <li>Auto formatting—Not only does it call you out on formatting violations, it auto-fixes them</li> <li>Code style discussions disappear—just use whatever Black does</li> <li>Super easy to make several code bases look consistent (no more mental gymnastics to format new code to match its surroundings)</li> <li>Automatically enforce consistent formatting on CI with “black --check” (I use a combo of flake8 + black because flake8 also catches syntax errors and some other “code smells”) <ul> <li>pro-tip: set up a pre-commit hook/rule to automatically run black before committing to Git. Also add it to your editor workflow (reformat on save / reformat on paste)</li> </ul></li> </ul></li> <li>Tool support: <ul> <li><a href="">Built into</a> the Python extension for VS Code (which Łukasz uses 😉)</li> <li><a href="">Plug-in</a> for PyCharm (for Michael and Brian 😁 )</li> <li>Support in <a href="">pre-commit</a></li> </ul></li> <li>For the most part I really like the formatting Black applies, if you’re not a fan you might hate this tool because it makes your code look “ugly” 🙂 </li> <li>Still in beta but found it very useful and helpful as of October 2018. Give it a try!</li> </ul> <p><strong>Brett Cannon #5:</strong> <a href=""><strong>A Web without JavaScript</strong></a>: Russell Keith-Magee at PyCon AU</p> <ul> <li>JavaScript has a monopoly in web browsers for client-side programming</li> <li>Mono-language situations are not good for anyone</li> <li>Can Python somehow break into the client-side web world?</li> <li>Example implementation of Luhn algorithm: <ul> <li>JavaScript: 0.4KB</li> <li><a href="">Transcrypt</a>: transpile to 32KB</li> <li><a href="">Brython</a>: Python compiler for 0.5KB + 646KB bootstrap</li> <li><a href="">Batavia</a>: Eval loop for 1.2KB + 5MB bootstrap</li> <li><a href="">Pyodide</a>: CPython compiled to WASM for 0.5KB + 3MB bootstrap</li> </ul></li> <li>WASM as a Python target might make this feasible <ul> <li>Example written in C compiled to 22KB (w/ a 65KB bootstrap for older browsers)</li> </ul></li> <li>Maybe easier to target Electron/Node instead of client-side web initially?</li> <li>Scott Hanselman’s blog post <a href=""></a></li> <li>Hanselminutes interview <a href=""></a></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Async WebDriver implementation for asyncio and asyncio-compatible frameworks</strong></a></p> <ul> <li>You’ve heard of Selenium but in an async world what do we use? Answer: <code>arsenic</code></li> </ul> <pre><code> # Example: Let's run a local Firefox instance. async def example(): # Runs geckodriver and starts a firefox session async with get_session(Geckodriver(), Firefox()) as session: # go to await session.get('') # wait up to 5 seconds to get the h1 element from the page h1 = await session.wait_for_element(5, 'h1') # print the text of the h1 element print(await h1.get_text()) </code></pre> <ul> <li>Use cases include testing of web applications, load testing, automating websites, web scraping or anything else you need a web browser for. </li> <li>It uses real web browsers using the Webdriver specification.</li> <li>Warning: While this library is asynchronous, web drivers are not. You must call the APIs in sequence. The purpose of this library is to allow you to control multiple web drivers asynchronously or to use a web driver in the same thread as an asynchronous web server.</li> <li><a href="">Arsenic with pytest</a></li> <li>Supported browsers <ul> <li><a href="">Headless Google Chrome</a></li> <li><a href="">Headless Firefox</a></li> </ul></li> <li>Everyone’s thoughts on async in Python these days?</li> <li>Selenium-Grid <a href=""></a></li> </ul> <p><strong>Extra:</strong> </p> <ul> <li>Take the python survey: <a href=""></a> </li> <li>3.7.1rc1 is out <a href=""></a> </li> <li>A good review on Python packaging <a href=""></a></li> <li>New September release of <a href="">Python Extension for Visual Studio Code</a> — lots of new features, like automatic environment activation in the terminal, debugging improvements, and more!</li> <li><a href="">Submit a talk</a> to <a href="">PyCascades</a> happening February 2019 in Seattle. <a href="">Call for proposals</a> closes October 21st. Mentorship available.</li> </ul>
Oct 19, 2018
#99 parse - the regex antidote in Python
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Forbes cyber article</strong>: <a href=""><strong>Cyber Saturday—Doubts Swirl Around Bloomberg's China Chip Hack Report</strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>parse</strong></a></p> <ul> <li>“<em>parse() is the opposite of format()</em><em>”</em></li> <li><code>regex</code> not required for parsing strings.</li> <li>Provides these functionalities: export <code>parse()</code>, <code>search()</code>, <code>findall()</code>, and <code>with_pattern()</code></li> </ul> <pre><code> # Note: space around &lt; p &gt; etc added to escape markdown parser safety measures &gt;&gt;&gt; parse("It's {}, I love it!", "It's spam, I love it!") &lt; Result ('spam',) {} &gt; &gt;&gt;&gt; search('Age: {:d}\n', 'Name: Rufus\nAge: 42\nColor: red\n') ( Result (42,) {} ) &gt;&gt;&gt; ''.join(r.fixed[0] for r in findall("&lt;{}&gt;", "\&lt; p &gt;the &lt; b &gt;bold&lt; /b &gt; text&lt; /p &gt;")) 'the bold text' </code></pre> <ul> <li>Can also compile for repeated use.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>fman Build System</strong></a></p> <ul> <li>FBS lets you create GUI apps for Windows, Mac and Linux </li> <li>via Michael Herrmann</li> <li>Build Python GUIs, with Qt – in minutes</li> <li>Write a desktop application with PyQt or Qt for Python. </li> <li>Use fbs to package and deploy it on Windows, Mac and Linux. </li> <li>Avoid months of painful work with the proven solutions provided by fbs.</li> <li>Easy Packaging: Unlike other solutions, fbs makes packaging easy. Create installers for your app in seconds and distribute them to your users – on Windows, Mac and Linux!</li> <li>Open Source: fbs's source code is available on GitHub. You can use it for free in open source projects licensed under the GPL. Commercial licenses are also offered. <ul> <li>Free under the GPL. If that's too restrictive, a commercial license is 250 Euros once.</li> <li>PyQt's licensing is similar (GPL/Commercial). A license for it is € 450 (<a href="">source</a>).</li> </ul></li> <li>Came from fman, a dual-pane file manager for Mac, Windows and Linux</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>fastjsonschema</strong></a></p> <ul> <li>Validate JSON against a schema, quickly.</li> </ul> <p><a href=""></a></p> <p><strong>Michael #4:</strong> <a href=""><strong>IPython 7.0, Async REPL</strong></a></p> <ul> <li>via <a href="">Nick Spirit</a></li> <li>Article by <a href="">Matthias Bussonnier</a></li> <li>We are pleased to announce the release of <a href="">IPython 7.0, the powerful Python interactive shell</a> that goes above and beyond the default Python REPL with advanced tab completion, syntactic coloration, and more.</li> <li>Not having to support Python 2 allowed us to make full use of new Python 3 features and bring never before seen capability in a Python Console, see the <a href="">Python 3 Statement.</a></li> <li>One of the core features we focused on for this release is the ability to (ab)use the <em>async</em> and <em>await</em> syntax available in Python 3.5+.</li> <li>TL;DR: You can now use <em>async</em>/<em>await</em> at the top level in the IPython terminal and in the notebook, it should — in most of the cases — “just work”.</li> <li>The only thing you need to remember is: If it is an async function you need to await it.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>molten</strong></a></p> <p><strong>Michael #6:</strong> <a href=""><strong>A Python love letter</strong></a></p> <ul> <li>Dear Python, where have you been all my life? (reddit thread)</li> <li>I am NOT a developer. But, I've tinkered with programming (in BASIC, Visual Basic, Perl, now Python) when needed over the years</li> <li>I decided that I needed to script something, and hoped that learning how to do it in Python was going to take me significantly less time than doing it manually - with the benefit of future timesavings. No, I didn't go from 0 to production in a day. But if my coworkers will leave me alone, I might be in production by the end of the day tomorrow.</li> <li>What I'm working on today isn't super complex — But putting together what I've done so far has just been a complete joy.</li> <li>Overall it feels natural, intuitive, and relatively easy to understand and write the code for the basic things I'm doing - I haven't had this much fun doing stuff with code since the days fooling around with BASIC in my teens.</li> <li>Feedback / comments <ul> <li>Welcome to the club. I came up on c++; my job highly trained me in C and assembly but every project I touch I think, wait, "we can do 95% this in python". And we do.</li> <li>I used to have a chip on my shoulder. I wanted to do things the hard way to truly understand them. I went with C++. … I learned that doing things the smart way was better than doing things the hard way and didn't interfere with learning.</li> <li>I felt the exact same way I finally decided to learn it. It's like a breath of fresh air. Sadly there are few things in my life that made me feel like this, Python and Bitcoin both give me the same levels of enjoyment. … I've used Java, Groovy, Scala, Objective-C, C, C++, C#, Perl and Javascript in a professional capacity over the years and nothing feels as natural to me as Python does. The developers truly deserve any donations they get for making it. … Hell <strong>my next two planned tattoos are bitcoin and python</strong> logos on my wrists.</li> <li>I taught myself Python a little over 3 years ago and I quickly went from not being programmer to being a programmer. … However the real popularity of Python comes from the depth and quality of 3rd party libraries and how easy they are to install. </li> </ul></li> </ul> <p><strong>Extra:</strong> </p> <ul> <li><a href=""><strong>Brian: Power Mode II</strong></a></li> </ul>
Oct 16, 2018
#98 Python-Electron as a Python GUI
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Making Etch-a-Sketch Art With Python</strong></a></p> <ul> <li>Really nice write up of methodically solving problems with simplifying the problem space, figuring out what parts need solved, grabbing off the shelf bits that can help, and putting it all together. </li> <li>Plus it would be a fun weekend (or several) project with kids helping.</li> <li>Controlling the Etch-a-Sketch <ul> <li>Raspberry Pi, motors, cables, wood fixture</li> <li>Software to control the motors</li> </ul></li> <li>Picture simplification with edge detection with Canny edge detection.</li> <li>Lines to motor control with path finding with <a href="">networkx</a> library.</li> <li>Example results included in article.</li> <li>Pentium song:</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Dropbox moves to Python 3</strong></a></p> <ul> <li>They just rolled out one of the largest Python 3 migrations ever</li> <li>Dropbox is one of the most popular desktop applications in the world</li> <li>Much of the application is written using Python. In fact, Drew’s very first lines of code for Dropbox were written in Python for Windows using venerable libraries such as pywin32.</li> <li>Though we’ve relied on Python 2 for many years (most recently, we used Python 2.7), we began moving to Python 3 back in 2015.</li> <li>If you’re using Dropbox today, the application is powered by a Dropbox-customized variant of Python 3.5. </li> <li>Why Python 3? <ul> <li>Exciting new features: Type annotations and async &amp; await</li> <li>Aging toolchains: As Python 2 has aged, the set of toolchains initially compatible for deploying it has largely become obsolete</li> </ul></li> <li>Embedding Python <ul> <li>To solve build and deploy problem, we decided on a new architecture to embed the Python runtime in our native application.</li> <li>Deep integration with the OS (e.g. smart sync) means native apps are required</li> </ul></li> <li>In future posts, we’ll look at: <ul> <li>How we report crashes on Windows and macOS and use them to debug both native and Python code.</li> <li>How we maintained a hybrid Python 2 and 3 syntax, and what tools helped.</li> <li>Our very best bugs and stories from the Python 3 migration.</li> </ul></li> </ul> <p><strong>Brian #3: Resources for PyCon that relate to really any talk venue</strong></p> <ul> <li><a href="">Speaking page</a></li> <li><a href="">Talk proposal tips and resources</a></li> <li>And the <a href="">poster session</a>. Way cooler than I originally understood.</li> <li><a href="">Mariatta recently published her set of proposals</a> <ul> <li>Nice clean examples that don’t look overwhelming</li> <li>There’s also some links to examples at the talk proposal page.</li> </ul></li> <li>Related, on attending PyCon (or other technical conferences): <ul> <li><a href="">You don't need to be a Pro @ Python to crack the code of Pycon</a> <ul> <li>missing: hang out and talk with, ask questions, and possibly help out with communities as part of the Expo.</li> </ul></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Electron as GUI of Python Applications</strong></a></p> <ul> <li>via <a href="">Andy Bulka</a></li> <li><a href=""><strong>Electron Python</strong></a> is a template of code where you use <a href="">Electron</a> (nodejs + chromium) as a GUI talking to Python 3 as a backend via zerorpc. Similar to <a href="">Eel</a> but much more capable e.g. you get proper native operating system menus — and users don’t need to have Chrome already installed.</li> <li>Needs to run zerorpc server and then start electron separately — can be done via the node backend</li> <li>using Electron as a GUI toolkit gets you <ul> <li>native menus, notifications</li> <li>installers, automatic updates to your app</li> <li>debugging and profiling that you are used to, using the Chrome debugger</li> <li>ES6 syntax (a cleaner Javascript with classes, module imports, no need for semicolons etc.). Squint, look sideways, and it kinda looks like Python… ;-)</li> <li>the full power of nodejs and its huge npm package repository</li> <li>the large community and ecosystem of Electron</li> </ul></li> <li>How to package this all?</li> <li><a href=""><strong>Building a deployable Python-Electron App</strong></a> post by Andy Bulka <ul> <li>One of the great things about using Electron as a GUI for Python is that you get to use cutting edge web technologies and you don’t have to learn some old, barely maintained GUI toolkit</li> <li>How much momentum, money, time and how many developer minds are focused on advancing web technologies? Answer: it’s staggeringly huge. </li> <li>Compare this with the number of people maintaining old toolkits from the 90’s e.g. wxPython? Answer: perhaps one or two people in their spare time. </li> <li>Which would you rather use?</li> <li>Final quote: And someone please wrap Electron-Python into an IDE so that in the future all we have to do is click a ‘build’ button — like we could 20 years ago. :-)</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>pluggy: A minimalist production ready plugin system</strong></a> </p> <ul> <li><a href="">docs</a></li> <li>plugin management and hook system used by pytest</li> <li>A separate package to allow other projects to include plugin capabilities without exposing unnecessary state or behavior of the host project.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>How China Used a Tiny Chip to Infiltrate U.S. Companies</strong></a></p> <ul> <li>via <a href="">Eduardo Orochena</a></li> <li>The attack by Chinese spies reached almost 30 U.S. companies, including Amazon and Apple, by compromising America’s technology supply chain, according to extensive interviews with government and corporate sources.</li> <li>In 2015, Inc. began quietly evaluating a startup called Elemental Technologies, a potential acquisition to help with a major expansion of its streaming video service, known today as Amazon Prime Video. (from Portland!)</li> <li>To help with due diligence, AWS, which was overseeing the prospective acquisition, hired a third-party company to scrutinize Elemental’s security</li> <li>servers were assembled for Elemental by <a href="">Super Micro Computer Inc.</a>, a San Jose-based company (commonly known as Supermicro) that’s also one of the world’s biggest suppliers of server motherboards</li> <li>Nested on the servers’ motherboards, the testers found a tiny microchip, not much bigger than a grain of rice, that wasn’t part of the boards’ original design.</li> <li>Amazon reported the discovery to U.S. authorities, sending a shudder through the intelligence community. Elemental’s servers could be found in Department of Defense data centers, the CIA’s drone operations, and the onboard networks of Navy warships. And Elemental was just one of hundreds of Supermicro customers.</li> <li>During the ensuing top-secret probe, which remains open more than three years later, investigators determined that the chips allowed the attackers to create a stealth doorway into any network that included the altered machines. Multiple people familiar with the matter say investigators found that the chips had been inserted at factories run by manufacturing subcontractors in China.</li> <li>One government official says China’s goal was long-term access to high-value corporate secrets and sensitive government networks. No consumer data is known to have been stolen.</li> <li>American investigators eventually figured out who else had been hit. Since the implanted chips were designed to ping anonymous computers on the internet for further instructions, operatives could hack those computers to identify others who’d been affected.</li> </ul> <p><strong>Extra:</strong> </p> <ul> <li>Michael's Async course <a href=""><strong></strong></a></li> </ul>
Oct 08, 2018
#97 Java goes paid
<p>Sponsored by DataDog -- <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Making a PyPI-friendly README</strong></a></p> <ul> <li>twine now checks for rendering problems with README</li> <li>Install the latest version of <a href="">twine</a>; version 1.12.0 or higher is required: <code>pip install --upgrade twine</code></li> <li>Build the sdist and wheel for your project as described under <a href="">Packaging your project</a>.</li> <li>Run <code>twine check</code> on the sdist and wheel: <code>twine check dist/*</code></li> <li>This command will report any problems rendering your README. If your markup renders fine, the command will output <code>Checking distribution FILENAME: Passed</code>.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Java goes paid</strong></a></p> <ul> <li>Oracle's new Java SE subs: Code and support for $25/processor/month</li> <li>Prepare for audit after inevitable change, says Oracle licensing consultant</li> <li>There’s also a little bit of stick to go with the carrot, because come January 2019 Java SE 8 on the desktop won’t be updated any more … unless you buy a sub.</li> <li>The short version is that every commercial enterprise needs to look at their Java SE (Standard Edition) usage to see if they need to do something with licensing.</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Absolute vs Relative Imports in Python</strong></a></p> <ul> <li>Review of how imports are used, along with subpackages and <code>from</code> <ul> <li>ex: <code>from package.sub import func</code></li> </ul></li> <li>Relative: what does this mean:</li> </ul> <pre><code>from .some_module import some_class from ..some_package import some_function from . import some_class </code></pre> <p><strong>Michael #4:</strong> <a href=""><strong>pyxel - A retro game engine for Python</strong></a> </p> <ul> <li>Thanks to its simple specifications inspired by retro gaming consoles, such as only 16 colors can be displayed and only 4 sounds can be played back at the same time, you can feel free to enjoy making pixel art style games.</li> <li>Run on Windows, Mac, and Linux</li> <li>Code writing with Python3</li> <li>After installing Pyxel, the examples of Pyxel will be copied to the current directory with the following command: <code>install_pyxel_examples</code> </li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Click 7.0 Released</strong></a></p> <ul> <li><a href="">Changelog</a></li> <li>Drop support for Python 2.6 and 3.3.</li> <li>Add native ZSH autocompletion support. </li> <li>Usage errors now hint at the <code>--help</code> option</li> <li>Really long list of changes since the last release at the beginning of 2017</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>How we spent 30k USD in Firebase in less than 72 hours</strong></a></p> <ul> <li>the largest crowdfunding campaign in Colombia, collecting 3 times more than the previous record so far in only two days!</li> <li>Run on the Vaki platform -- subject of this article</li> <li>We had reached more than 2 million sessions, more than 20 million pages visited and received more than 15 thousand supports. This averages to a thousand users active on the site in average and collecting more than 20 supports per minute.</li> <li>Site was running slow, tried things like upgraded the frontend frameworks </li> <li>Logged into Firebase: had spent $30,356.56 USD in just 72 hours! Going at $600/hr</li> <li>All came down to a very bad implementation of <code>this.loadPayments()</code>.</li> <li>Comments are interesting</li> <li>It could happen to any of us, it happened to me this month.</li> </ul> <p>Extras:</p> <ul> <li><a href=""><strong>Dropbox has upgraded from Python 2 → 3</strong></a>!</li> <li>Michael’s async course is live: <a href=""><strong>Async Techniques and Examples in Python</strong></a></li> <li><a href=""><strong>2019 PyCon CFPs open</strong></a></li> <li><a href=""><strong>PyCascades CFP is open until mid-Oct</strong></a></li> </ul>
Sep 28, 2018
#96 Python Language Summit 2018
<p>Sponsored by DigitalOcean -- <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Plumbum: Shell Combinators and More</strong></a></p> <ul> <li>Toolbox of goodies to do shell-like things from Python.</li> <li>“The motto of the library is <strong>“Never write shell scripts again”</strong>, and thus it attempts to mimic the <strong>shell syntax</strong> (<em>shell combinators</em>) where it makes sense, while keeping it all <strong>Pythonic and cross-platform</strong>.”</li> </ul> <p>Example:</p> <pre><code>&gt;&gt;&gt; from plumbum.cmd import grep, wc, cat, head &gt;&gt;&gt; chain = ls["-a"] | grep["-v", "\\.py"] | wc["-l"] &gt;&gt;&gt; print chain /bin/ls -a | /bin/grep -v '\.py' | /usr/bin/wc -l &gt;&gt;&gt; chain() u'13\n' &gt;&gt;&gt; ((cat &lt; "") | head["-n", 4])() u'#!/usr/bin/env python\nimport os\n\ntry:\n' &gt;&gt;&gt; (ls["-a"] &gt; "file.list")() u'' &gt;&gt;&gt; (cat["file.list"] | wc["-l"])() u'17\n' </code></pre> <p><strong>Michael #2:</strong> <a href="">Windows 10 Linux subsystem for Python developers</a></p> <ul> <li>via <a href="">Marcus Sherman</a></li> <li>“One of the hardest days in teaching introduction to bioinformatics material is the first day: Setting up your machine.”</li> <li>While I have seen a very large bias towards Macs in academia, there are plenty of people that keep their Windows machines as a badge of pride... Marcus included.</li> <li>Even though Anaconda is cross platform and helpful, how does this work on Windows? <ul> <li><code>python3 -m venv .env</code> and <code>source .env/bin/activate</code>?</li> <li>Spoiler alert: Not well.</li> </ul></li> <li>Step by step getting Ubuntu on Windows</li> <li>Shows how to setup an x-server </li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Type hints cheat sheet (Python 3)</strong></a></p> <ul> <li>Do you remember how to type hint duck types? <ul> <li>Something accessed like an array (list or tuple or …) and holds strings → <code>Sequence[str</code>]</li> <li>Something that works like a dictionary mapping integers to strings → <code>Mapping[int, str]</code></li> </ul></li> <li>As I’m adding more and more typing to interface functions, I keep this cheat sheet bookmarked.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Python driving new languages</strong></a></p> <ul> <li>Here are five predictions for what programming will look like 10 years from now. <ul> <li>Programming will be more abstract</li> <li>Trends like serverless technologies, containers, and low code platforms suggest that many developers may work at higher levels of abstraction in the future</li> <li>AI will become part of every developer's toolkit—but won't replace them</li> <li>A universal programming language will arise</li> <li>To reap the benefits of emerging technologies like AI, programming has to be easy to learn and easy to build upon</li> <li>"Python may be remembered as being the great-great-great grandmother of languages of the future, which underneath the hood may look like the English language, but are far easier to use,"</li> <li>Every developer will need to work with data</li> <li>Programming will be a core tenet of the education system</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>asyncio documentation rewritten from scratch</strong></a></p> <ul> <li><a href="">twitter thread</a> by <a href=""><strong>Yury Selivanov</strong></a><a href=""><strong>‏</strong></a> <ul> <li>“Big news! asyncio documentation has been rewritten from scratch! Read the new version here: <a href=""> …</a>. Huge thanks to <a href="">@WillingCarol</a>, <a href="">@elprans</a>, and <a href="">@andrew_svetlov</a> for support, ideas, and reviews!’</li> <li>“BTW, this is just the beginning. We'll continue to refine and update the documentation. Next up is adding two tutorials: one teaching high-level concepts and APIs, and another teaching how to use protocols and transports. A section about asyncio architecture is also planned.”</li> <li>“And this is just the beginning not only for asyncio documentation, but for asyncio itself. Just for Python 3.8 we plan to add: <ul> <li>new streaming API</li> <li>TaskGroups and cancel scopes</li> <li>Supervisors and tracing API</li> <li>new SSL implementation</li> <li>many usability improvements”</li> </ul></li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>The 2018 Python Language Summit</strong></a></p> <ul> <li>Here are the sessions: <ul> <li><a href="">Subinterpreter support for Python</a>: a way to have a better story for multicore scalability using an existing feature of the language. <ul> <li>Subinterpreters will allow multiple Python interpreters per process and there is the potential for zero-copy data sharing between them. </li> <li>But subinterpreters share the GIL, so that needs to be changed in order to make it multicore friendly.</li> </ul></li> <li><a href="">Modifying the Python object model</a>: looking at changes to CPython data structures to increase the performance of the interpreter. - via Instagram and Carl Shapiro - By modifying the Python object model fairly substantially, they were able to roughly double the performance - A little controversial - Shapiro's overall point was that he felt Python sacrificed its performance for flexibility and generality, but the dynamic features are typically not used heavily in performance-sensitive production workloads.</li> <li><a href="">A Gilectomy update</a>: a status report on the effort to remove the GIL from CPython. <ul> <li>Larry Hastings updated attendees on the status of his <a href="">Gilectomy</a> project.</li> <li>Since his <a href="">status report</a> at last year's summit, little has happened, which is part of why the session was so short. He hasn't given up on the overall idea, but it needs a new approach.</li> </ul></li> <li><a href="">Using GitHub Issues for Python</a>: a discussion on moving from to GitHub Issues. <ul> <li>Mariatta Wijaya described her reasoning for advocating moving Python away from its <a href="">current bug tracker</a> to GitHub Issues.</li> <li>it would complete Python's <a href="">journey to GitHub</a> that <a href="">started a ways back</a>.</li> </ul></li> <li><a href="">Shortening the Python release schedule</a>: a discussion on possibly changing from an 18-month to a yearly cadence. <ul> <li>The Python release cycle has an 18-month cadence; a new major release (e.g. Python 3.7) is made roughly on that schedule. </li> <li>But Łukasz Langa, who is the release manager for Python 3.8 and 3.9, would like to see things move more quickly—perhaps on a yearly cadence.</li> </ul></li> <li><a href="">Unplugging old batteries</a>: should some older, unloved modules be removed from the standard library? <ul> <li>Python is famous for being a "batteries included" language—its standard library provides a versatile set of modules with the language</li> <li>There may be times when some of those batteries have reached their end of life.</li> <li>Christian Heimes wanted to suggest a few batteries that may have outlived their usefulness and to discuss how the process of retiring standard library modules should work.</li> </ul></li> <li><a href="">Linux distributions and Python 2</a>: the end of life for Python 2 is coming, what distributions are doing to prepare. <ul> <li>Christian Heimes wanted to suggest a few batteries that may have outlived their usefulness and to discuss how the process of retiring standard library modules should work.</li> <li>To figure out how to help the Python downstreams so that Python 2 can be fully discontinued.</li> </ul></li> <li><a href="">Python static typing update</a>: a look at where static typing is now and where it is headed for Python 3.7. <ul> <li>Started things off by talking about <a href="">stub files</a>, which contain type information for libraries and other modules.</li> <li>Right now, static typing is only partially useful for large projects because they tend to use a lot of packages from the Python Package Index (PyPI), which has limited stub coverage. There are only 35 stubs for third-party modules in the <a href="">typeshed</a> library, which is Python's stub repository.</li> <li>He suggested that perhaps a centralized library for stubs is not the right development model. Some projects have stubs that live outside of typeshed, such as Django and SQLAlchemy.</li> <li><a href="">PEP 561</a> ("Distributing and Packaging Type Information") will provide a way to pip install stubs from packages that advertise that they have them.</li> </ul></li> <li><a href="">Python virtual environments</a>: a short session on virtual environments and ideas for other ways to isolate local installations. <ul> <li>Steve Dower brought up the shortcomings of Python <a href="">virtual environments</a>, which are meant to create isolated installations of the language and its modules.</li> <li>Thomas Wouters defended virtual environments in a response: The correct justification is that for the average person, not using a virtualenv all too soon creates confusion, pain, and very difficult to fix breakage. Starting with a virtualenv is the easiest way to avoid that, at <em>very</em> little cost.</li> <li>But Beazley and others (including Dower) think that starting Python tutorials or training classes with a 20-minute digression on setting up a virtual environment is wasted time. </li> </ul></li> <li><a href="">PEP 572 and decision-making in Python</a>: a discussion of the controversy around PEP 572 and how to avoid the thread explosion that it caused in the future. <ul> <li>The "PEP 572 mess" was the topic of a 2018 Python Language Summit session led by benevolent dictator for life (BDFL) Guido van Rossum.</li> </ul></li> <li><a href="">Getting along in the Python community</a>: trying to find ways to keep the mailing list welcoming even in the face of rudeness. <ul> <li>About tkinter…</li> </ul></li> <li><a href="">Mentoring and diversity for Python</a>: a discussion on how to increase the diversity of the core development team. <ul> <li>Victor Stinner outlined some work he has been doing to mentor new developers on their path toward joining the core development ranks</li> <li>Mariatta Wijaya gave a very personal talk that described the diversity problem while also providing some concrete action items that the project and individuals could take to help make Python more welcoming to minorities.</li> </ul></li> </ul></li> </ul> <p><strong>Extras</strong></p> <p>Listener feedback: CUDA is NVidia only, so no MacBook pro unless you have a custom external GPU.</p>
Sep 22, 2018
#95 Unleash the py-spy!
<p>Sponsored by DataDog -- <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>dataset: databases for lazy people</strong></a></p> <ul> <li><strong>dataset</strong> provides a simple abstraction layer removes most direct SQL statements without the necessity for a full ORM model - essentially, databases can be used like a JSON file or NoSQL store.</li> <li>A simple data loading script using <strong>dataset</strong> might look like this:</li> </ul> <pre><code> import dataset db = dataset.connect('sqlite:///:memory:') table = db['sometable'] table.insert(dict(name='John Doe', age=37)) table.insert(dict(name='Jane Doe', age=34, gender='female')) john = table.find_one(name='John Doe') </code></pre> <p><strong>Michael #2:</strong> <a href=""><strong>CuPy GPU NumPy</strong></a></p> <ul> <li>A NumPy-compatible matrix library accelerated by CUDA</li> <li>How many cores does a modern GPU have?</li> <li>CuPy's interface is highly compatible with NumPy; in most cases it can be used as a drop-in replacement.</li> <li>You can easily make a custom CUDA kernel if you want to make your code run faster, requiring only a small code snippet of C++. CuPy automatically wraps and compiles it to make a CUDA binary</li> <li>PyCon 2018 presentation: <a href="">Shohei Hido - CuPy: A NumPy-compatible Library for GPU</a></li> <li>Code example</li> </ul> <pre><code> &gt;&gt;&gt; # This will run on your GPU! &gt;&gt;&gt; import cupy as np # This is the only non-NumPy line &gt;&gt;&gt; x = np.arange(6).reshape(2, 3).astype('f') &gt;&gt;&gt; x array([[ 0., 1., 2.], [ 3., 4., 5.]], dtype=float32) &gt;&gt;&gt; x.sum(axis=1) array([ 3., 12.], dtype=float32) </code></pre> <p><strong>Brian #3:</strong> <a href=""><strong>Automate Python workflow using pre-commits</strong></a></p> <ul> <li>We covered pre-commit in <a href="">episode 84</a>, but I still had trouble getting my head around it.</li> <li>This article by LJ Miranda does a great job with the workflow introduction and configuration necessary to get pre-commit working for black and flake8.</li> <li>Includes a nice visual of the flow.</li> <li>Demo of it all in action with a short video.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>py-spy</strong></a></p> <ul> <li>Sampling profiler for Python programs </li> <li>Written by <a href="">Ben Frederickson</a></li> <li>Lets you visualize what your Python program is spending time on without restarting the program or modifying the code in any way.</li> <li>Written in Rust for speed</li> <li>Doesn't run in the same process as the profiled Python program</li> <li>Does NOT it interrupt the running program in any way.</li> <li>This means Py-Spy is safe to use against production Python code.</li> <li>The default visualization is a top-like live view of your python program</li> <li><a href=""><strong>How does py-spy work?</strong></a> Py-spy works by directly reading the memory of the python program using the <a href="">process_vm_readv</a> system call on Linux, the <a href="">vm_read</a> call on OSX or the <a href="">ReadProcessMemory</a> call on Windows.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>SymPy is a Python library for symbolic mathematics</strong></a></p> <ul> <li>“Symbolic computation deals with the computation of mathematical objects symbolically. This means that the mathematical objects are represented exactly, not approximately, and mathematical expressions with unevaluated variables are left in symbolic form.”</li> <li>example:</li> </ul> <pre><code> &gt;&gt;&gt; integrate(sin(x**2), (x, -oo, oo)) √2⋅√π ───── 2 </code></pre> <ul> <li>examples on site are interactive so you can play with it without installing anything.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Starlette ASGI web framework</strong></a></p> <ul> <li>The little ASGI framework that shines.</li> <li>It is ideal for building high performance asyncio services, and supports both HTTP and WebSockets.</li> <li>Very flask-esq</li> <li>Can use <ul> <li><a href="">ultrajson - Ultra fast JSON decoder and encoder written in C with Python bindings</a></li> <li>aiofiles for file responses</li> </ul></li> <li>Run using uvicorn</li> </ul> <p>Extras:</p> <p><strong>Michael:</strong> <a href=""><strong>PyCon 2019</strong></a> <strong>dates out, put them on your calendar!</strong></p> <ul> <li>Tutorials: May 1-2 • Wednesday, Thursday</li> <li>Talks and Events: May 3–5 • Friday, Saturday, Sunday</li> <li>Sprints: May 6–9 • Monday through Thursday</li> </ul> <p><strong>Listener follow up on git pre-commit hooks util:</strong> <a href=""><strong>pre-commit package</strong></a></p> <ul> <li>Matthew Layman, <a href="">@mblayman</a></li> <li>Heard the discussion about Git commit hooks at the end. I wanted to bring up pre-commit as an interesting project (written in Python!) that's useful for Git commit hooks.</li> <li>tl;dr: <ul> <li>$ pip install pre-commit</li> <li>$ ... create a .pre-commit-config.yaml</li> <li>$ pre-commit install # This is a one time operation.</li> </ul></li> <li>pre-commit's job is to manage a project's Git commit hooks. We use this on my team at work and the devs only need to run <code>pre-commit install</code>. This saves us from a bunch of failing CI builds where flake8 or other code style checks would fail.</li> <li>We use pre-commit to run flake8 and black before allowing a commit to proceed. Some projects have a pre-commit configuration to use right out of the box (e.g., black <a href=""></a>).</li> </ul> <p><strong>Listener: You don't need that (pattern)</strong></p> <ul> <li>John Tocher</li> <li>PyCon AU <a href="">Talk Called "You don't need that</a>” - by Christopher Neugebauer, it was an interesting take on why with a modern and powerful language like python, you may not need the conventionally described design patterns, ala the "Gang of four".</li> </ul>
Sep 15, 2018
#94 Why don't you like notebooks?
<p>Sponsored by DigialOcean -- <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Python Patterns</strong></a></p> <ul> <li><a href="">@brandon_rhodes</a> vs GOF</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Arctic: Millions of rows a sec (time data)</strong></a></p> <ul> <li>Arctic is a high-performance datastore for numeric data. It supports Pandas, numpy arrays and pickled objects out-of-the-box, with pluggable support for other data types and optional versioning.</li> <li>Arctic can query millions of rows per second per client, achieves ~10x compression on network bandwidth, ~10x compression on disk, and scales to hundreds of millions of rows per second per MongoDB instance.</li> <li>Arctic has been under active development at Man AHL since 2012.</li> <li>Super fast, some latency numbers: <ul> <li>1xDay Data 4ms for 10k rows, vs 2,210 ms from SQL Server)</li> <li>Tick Data 1s for 3.5 MB (Python) or 15 MB (Java) vs 15-40sec from “other tick”</li> </ul></li> <li>Versioned data</li> <li>Built on MongoDB</li> <li><a href="">Slides</a></li> <li>Based on pandas</li> <li>Tested with pytest</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>PyCon Australia videos</strong></a></p> <ul> <li><a href=";t=0s&amp;list=PLs4CJRBY5F1KrUr7z_2mur2QdAKXyh-k3&amp;index=39">How To Publish A Package On PyPI</a> <ul> <li>Mark Smith <a href="">@judy2k</a> <a href=";t=0s&amp;list=PLs4CJRBY5F1KrUr7z_2mur2QdAKXyh-k3&amp;index=39"></a></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>GAE: Introducing App Engine Second Generation runtimes and Python 3.7</strong></a></p> <ul> <li>Today, Google Cloud is announcing the availability of Second Generation App Engine standard runtimes, a significant upgrade to the platform that allows you to easily run web apps using up-to-date versions of popular languages, frameworks and libraries.</li> <li>Python 3.7 is one of the new Second Generation runtimes that we <a href="">announced at Cloud Next</a>. </li> <li>Based on technology from the <a href="">gVisor container sandbox</a>, these Second Generation runtimes eliminate many previous App Engine restrictions, giving you the ability to write portable web apps and microservices that take advantage of App Engine's unique auto-scaling, built-in security and pay-per-use billing model.</li> <li>This new runtime allows you to take advantage of Python's vibrant ecosystem of open-source libraries and frameworks. While the Python 2 runtime only allowed the use of specific versions of whitelisted libraries, Python 3 supports arbitrary third-party libraries, including those that rely on C code and native extensions. Just add <a href="">Django 2.0</a>, <a href="">NumPy</a>, <a href="">scikit-learn</a> or your library of choice to a <code>requirements.txt</code> file. App Engine will install these libraries in the cloud when you deploy your app.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>I don’t like notebooks</strong></a></p> <ul> <li><a href="">@joelgrus</a></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>PEP 8000 -- Python Language Governance Proposal Overview</strong></a></p> <ul> <li>This PEP provides an overview of the selection process for a new model of Python language governance in the wake of <a href="">Guido's retirement</a>. Once the governance model is selected, it will be codified in <a href="">PEP 13</a>.</li> <li>PEPs in the lower 8000s describe the general process for selecting a governance model. <ul> <li><a href="">PEP 8001</a> - Python Governance Voting Process</li> <li><a href="">PEP 8002</a> - Open Source Governance Survey</li> </ul></li> <li>PEPs in the 8010s describe the actual proposals for Python governance. <ul> <li><a href="">PEP 8010</a> - The BDFL Governance Model</li> <li><a href="">PEP 8011</a> - The Council Governance Model</li> <li><a href="">PEP 8012</a> - The Community Governance Model</li> </ul></li> </ul> <p>Extras</p> <ul> <li>Free Brian Granger <a href="">ACM webcast</a> on Jupyter Friday</li> <li>TIOBE jump to #3:</li> </ul>
Sep 06, 2018
#93 Looking like there will be a PyBlazor!
<p>Sponsored by DataDog -- <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Replacing Bash Scripting with Python</strong></a>.</p> <ul> <li>reading &amp; writing files</li> <li>CLI’s and working with stdin, stdout, stderr</li> <li>Path and shutil</li> <li>replacing sed, grep, awk, with regex</li> <li>running processes</li> <li>dealing with datetime</li> <li>see also: <ul> <li><a href="">regex search and replace example scripts</a></li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>pyodide</strong></a></p> <ul> <li>Scientific Python in the browser <ul> <li><em>ALL</em> of CPython (allowed in the browser)</li> <li>NumPy</li> <li>MatPlotLib</li> <li>...</li> </ul></li> <li>Project by Mozilla</li> <li>We asked “<a href="">Will there be a PyBlazor?</a>” just two weeks ago. I think we are on a path…</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>The subset of reStructuredText worth committing to memory</strong></a></p> <ul> <li>A lot of Python packages document with reStructuredText, a lot of reStructuredText tutorials are overwhelming. This post is the answer.</li> <li>paragraphs are with two newlines</li> <li>headings use a weird underlined method of above and below and =, -, and ~</li> <li>bulleted lists work with asterisks but spacing is important</li> <li>italics and bold are with one or two surrounding asterisks</li> <li>inline code uses two backticks</li> <li>links and code snippets are weird and I have to always look this up, as with images, and internal references.</li> <li>so I’ll bookmark this link</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>bandit</strong></a></p> <ul> <li>via Anthony Shaw</li> <li>Bandit is a tool designed to find common security issues in Python code. </li> <li>To do this Bandit processes each file, builds an AST from it, and runs appropriate plugins against the AST nodes. Once Bandit has finished scanning all the files it generates a report.</li> <li>Issues detected: <ul> <li>B312 telnetlib</li> <li>B307 eval</li> <li>B110 try_except_pass</li> <li>B602 subprocess_popen_with_shell_equals_true</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Learn Python 3 within Jupyter Notebooks</strong></a></p> <ul> <li>just fun</li> <li>Also shows how to run <code>pytest</code> in a cell.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>detect-secrets</strong></a></p> <ul> <li>An enterprise friendly way of detecting and preventing secrets in code. </li> <li>From Yelp</li> <li>detect-secrets is an aptly named module for (surprise, surprise) detecting secrets within a code base.</li> <li>However, unlike other similar packages that solely focus on finding secrets, this package is designed with the enterprise client in mind: providing a <strong>backwards compatible</strong>, systematic means of: <ol> <li>Preventing new secrets from entering the code base,</li> <li>Detecting if such preventions are explicitly bypassed, and</li> <li>Providing a checklist of secrets to roll, and migrate off to a more secure storage.</li> </ol></li> <li>Allows you to set a baseline</li> <li>set it up as a git commit hook</li> </ul>
Aug 31, 2018
#92 Will your Python be compiled?
<p>Sponsored by Digital Ocean -- <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>IEEE Survey Ranks Programming Languages</strong></a></p> <ul> <li>via Martin Rowe, <a href="">@measureentblue</a></li> <li>Python on top. Was last year also, but this year it’s on top even for embedded.</li> <li>Some people dispute the numbers but I believe it.</li> <li>Projects contributing to the rise of Python in embedded: <ul> <li><a href="">MicroPython</a></li> <li><a href="">CircuitPython</a></li> <li><a href="">micro:bit</a></li> <li><a href="">Mu</a></li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>MyPyC</strong></a></p> <ul> <li>Thread on Python-Dev: Use of Cython</li> <li><em>It'd be *really</em> nice to at least be able to write some of the C API tests directly in Cython rather than having to fiddle about with splitting the test between the regrtest parts that actually define the test case and the extension module parts that expose the interfaces that we want to test.*</li> <li>Later in the thread, Yury Selivanov dropped a bomb shell. <ul> <li>Speaking of which, Dropbox is working on a new compiler they call "mypyc".</li> <li><strong>mypyc will compile type-annotated Python code to an optimized C.</strong> </li> <li>Essentially, mypyc will be similar to Cython, but mypyc is a <em>subset of Python</em>, not a superset.</li> <li>Interfacing with C libraries can be easily achieved with cffi. Being a strict subset of Python means that mypyc code will execute just fine in PyPy. They can even apply some optimizations to it eventually, as it has a strict and static type system.</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Beyond Interactive: Notebook Innovation at Netflix</strong></a></p> <ul> <li>Netflix is doing some very cool things with Jupyter, and sharing much of it through open source projects.</li> <li>Netflix has growing their use of Jupyter notebooks for many data related roles: <ul> <li>business, data, &amp; quantitative analysts</li> <li>algorithm, analytics, &amp; data engineers</li> <li>data, machine learning, &amp; research scientists</li> </ul></li> <li>All of these roles have common needs that are solved by Jupyter and related projects: <ul> <li>data exploration, preparation, validation, and productionalization (is that a word?)</li> </ul></li> <li>To help solve their use cases and make notebooks even easier to use for everyone at Netflix, they’ve started many open source projects that can be used by non-Netflix folks as well: <ul> <li>“<a href=""><strong>nteract</strong></a> is a next-gen React-based UI for Jupyter notebooks.”</li> <li>“<a href=""><strong>Papermill</strong></a> is a library for parameterizing, executing, and analyzing Jupyter notebooks. “</li> <li>“<a href=""><strong>Commuter</strong></a> is a lightweight, vertically-scalable service for viewing and sharing notebooks.”</li> <li>“<a href=""><strong>Titus</strong></a> is a container management platform that provides scalable and reliable container execution and cloud-native integration with Amazon AWS. “</li> </ul></li> <li>There’s a follow-on post that discusses how Netflix is scheduling notebook execution: <a href="">Scheduling Notebooks</a></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>How to create a Windows Service in Python</strong></a></p> <ul> <li>We have spoken about how to <a href=""><strong>run Python script as systemd service</strong></a></li> <li>Here’s the Windows edition <ul> <li>Run Python code on boo</li> <li>When logged out or logged in as another user</li> <li>As a restricted or different account</li> </ul></li> <li>Based on <a href="">pywin32</a> (very little documentation)</li> <li>Derive from a given base class then override the three main methods: <ul> <li><code>def start(self)</code> : if you need to do something at the service initialization.</li> <li>A good idea is to put here the initialization of the running condition</li> <li><code>def stop(self)</code> : if you need to do something just before the service is stopped.</li> <li>A good idea is to put here the invalidation of the running condition</li> <li><code>def main(self)</code> : your actual run loop. Just create a loop based on your running condition</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>An Overview of Packaging for Python</strong></a></p> <ul> <li><a href="">Started from an essay by Mahmoud Hashemi</a>, <a href="">@mhashemi</a></li> <li>Now part of PyPA documentation <ul> <li>Different techniques and tools for different types of Python projects</li> <li>modules</li> <li>packages <ul> <li>source distributions</li> <li>wheels</li> <li>binary distributions</li> </ul></li> <li>applications <ul> <li>this is the hairy part where a bullet point summary just won’t be enough. :)</li> </ul></li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>PEP 505 -- None-aware operators</strong></a></p> <ul> <li>Several modern programming languages have so-called "null-coalescing" or "null- aware" operators, including C# and Swift. These operators provide syntactic sugar for common patterns involving null references.</li> <li>Why not Python?</li> <li>Two cases: <ul> <li>The "null-coalescing" operator: To replace inline conditionals such as this <code>value if value is not None else "MISSING"</code> can now be just <code>value ??</code> <code>"MISSING"</code></li> <li>The "null-aware member access" operator: Chain calls into a fluent interface without testing for None: <code>return user?.orders.first()?.name</code> would replace this</li> </ul></li> </ul> <pre><code> if user is None: return None first_order = user.orders.first() if first_order is None: return None return </code></pre> <p>Extras:</p> <ul> <li>PyCascades: <a href=""></a></li> <li>Test and Code episode with DHH: <a href=""> </a></li> </ul>
Aug 25, 2018
#91 Will there be a PyBlazor?
<p>Sponsored by Datadog <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>What makes the Python Cool</strong></a></p> <ul> <li>Shankar Jha</li> <li>“some of the cool feature provided by Python”</li> <li>The Zen of Python: <code>import this</code></li> <li>XKCD: <code>import antigravity</code></li> <li>Swapping of two variable in one line: <code>a, b = b, a</code></li> <li>Create a web server using one line: <code>python -m http.server 8000</code></li> <li><code>collections</code></li> <li><code>itertools</code></li> <li>Looping with index: <code>enumerate</code></li> <li>reverse a list: <code>list(reversed(a_list))</code></li> <li><code>zip</code> tricks</li> <li>list/set/dict comprehensions</li> <li>Modern dictionary</li> <li><code>pprint</code></li> <li><code>_</code> when in interactive REPL</li> <li>Lots of great external libraries</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Django 2.1 released</strong></a></p> <ul> <li><a href="">The release notes</a> cover the smorgasbord of new features in detail, the <a href="">model “view” permission</a> is a highlight that many will appreciate.</li> <li>Django 2.0 has reached the end of mainstream support. The final minor bug fix release (which is also a security release), <a href="">2.0.8</a>, was issued today.</li> <li>Features <ul> <li>model “view” feature: This allows giving users read-only access to models in the admin.</li> <li>The new <code>[ModelAdmin.delete_queryset()](</code> method allows customizing the deletion process of the “delete selected objects” action.</li> <li>You can now <a href="">override the default admin site</a>.</li> <li>Lots of ORM features</li> <li>Cache: The <a href="">local-memory cache backend</a> now uses a least-recently-used (LRU) culling strategy rather than a pseudo-random one.</li> <li>Migrations: To support frozen environments, migrations may be loaded from <code>.pyc</code> files.</li> <li>Lots more</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Awesome Python Features Explained Using Harry Potter</strong></a></p> <ul> <li>Anna-Lena Popkes</li> <li><a href="">Initial blog post</a></li> <li>100 Days of code, with a Harry Potter universe bent.</li> <li>Up to day 18 so far.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Executing Encrypted Python with no Performance Penalty</strong></a></p> <ul> <li>Deploying Python in production presents a large attack surface that allows a malicious user to modify or reverse engineer potentially sensitive business logic. </li> <li>This is worse in cases of distributed apps.</li> <li>Common techniques to protect code in production are <a href="">binary signing</a>, <a href="">obfuscation</a>, or <a href="">encryption</a>. But, these techniques typically assume that we are protecting either a single file (EXE), or a small set of files (EXE and DLLs).</li> <li>In Python signing is not an option and source code is wide open.</li> <li>requirements were threefold: <ol> <li>Work with the reference implementation of Python,</li> <li>Provide strong protection of code against malicious and natural threats,</li> <li>Be performant both in execution time and in stored space</li> </ol></li> <li>This led to a pure Python solution using authenticated cryptography.</li> <li>Created a <code>.pyce</code> file that is encrypted and signed</li> <li>Customized import statement to load and decrypt them</li> <li>Implementation has no overhead in production. This is due to Python's <a href="">in-memory bytecode cache</a>.</li> </ul> <p><strong>Brian #5:</strong> <strong>icdiff and pytest-icdiff</strong></p> <ul> <li><a href="">icdiff</a>: “Improved colored diff” <ul> <li>Jeff Kaufman</li> </ul></li> <li><a href="">pytest-icdiff</a>: “better error messages for assert equals in pytest” <ul> <li>Harry Percival</li> </ul></li> </ul> <p><strong>Michael #6: Will there be a PyBlazor?</strong></p> <ul> <li>The .NET guys, and Steve Sanderson in particular, are undertaking <a href=""><strong>an interesting project</strong></a> with WebAssembly.</li> <li><strong>WebAssembly</strong> (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable target for compilation of high-level languages like C/C++/Rust, enabling deployment on the web for client and server applications.</li> <li>Works in Firefox, Edge, Safari, and Chrome</li> <li>Their project, <a href=""><strong>Blazor</strong></a>, has nearly the entire .NET runtime (AKA the CLR) running natively in the browser via WebAssembly.</li> <li>This is notable because the CLR is basically pure C code. What else is C code? Well, CPython!</li> <li>Includes Interpreted and AOT mode: <ul> <li><strong>Ahead-of-time (AOT) compiled mode:</strong> In AOT mode, your application’s .NET assemblies are transformed to pure WebAssembly binaries at build time.</li> </ul></li> <li>Being able to run .NET in the browser is a good start, but it’s not enough. To be a productive app builder, you’ll need a coherent set of standard solutions to standard problems such as UI composition/reuse, state management, routing, unit testing, build optimization, and much more.</li> <li>Mozilla called for this to exist for Python, but sadly didn’t contribute or kick anything off at PyCon 2018: <a href=""></a></li> <li>Gary Bernhardt’s <a href="">Birth and Death of JavaScript</a> video is required pre-reqs as well (<code>asm.js</code>).</li> </ul> <p>Extras and personal info:</p> <p>Michael:</p> <ul> <li><a href=""><strong>Building data-driven web apps</strong></a> course is being well received</li> <li><a href=";"><strong>Guido van Rossum: Python 3 retrospective</strong></a> — Guido’s final presentation as BDFL</li> </ul>
Aug 15, 2018
#90 A Django Async Roadmap
<p>Sponsored by Digital Ocean: <a href=""></a></p> <p>Brian #1: <a href="">Reproducible Data Analysis in Jupyter</a></p> <ul> <li>Amazing series of videos by Jake Vanderplas</li> <li>Exploring a data set through visualization in a Jupyter notebook</li> <li>There’s a lot of dense material there, from saving datasets to files, plotting in the notebook as opposed to outside in a separate window, using resampling, … </li> </ul> <p>Michael #2: <a href="">PySimpleGUI - For simple Python GUIs</a></p> <ul> <li>Via Mike Barnett</li> <li>Looking to take your Python code from the world of command lines and into the convenience of a GUI? </li> <li>Have a Raspberry Pi with a touchscreen that's going to waste because you don't have the time to learn a GUI SDK? </li> <li>Look no further, you've found your GUI package.</li> <li>Based on tkinter</li> <li>No dependencies (outside of Python itself): <code>pip install PySimpleGUI</code></li> <li>Python3 is required to run PySimpleGUI. It takes advantage of some Python3 features that do not translate well into Python2.</li> <li>Looking to help? → Port to other graphic engines. Hook up the front-end interface to a backend other than tkinter. Qt, WxPython, etc.</li> </ul> <p>Brian #3: <a href=""><strong>Useful tricks you might not know about Git stash</strong></a></p> <ul> <li><code>git stash save</code> - Stash the changes in a dirty working directory away</li> <li><code>git stash apply</code> - re-applies your changes after you do whatever you need to to your directory, like perhaps pull.</li> <li>Lots of neat things to do with stash <ul> <li>you can add a message so the stashed content has a nice label</li> <li><code>-u</code> will include untracked files when saving.</li> <li><code>git stash branch [HTML_REMOVED] stash@{1}</code> will create a new branch with the latest stash, and then deletes the latest stash</li> <li>Lots of other nice tricks in the article</li> </ul></li> <li>See also: <a href="">git-stash in git-scm book</a></li> </ul> <p>Michael #4: <a href="">A Django Async Roadmap</a></p> <ul> <li>via Andrew Godwin, from Django Channels</li> <li>Thinks that the time has come to start talking seriously about bringing async functionality into Django itself</li> <li>Open for public feedback</li> <li>The goal is to make Django a world-class example of what async can enable for HTTP requests, such as: <ul> <li>Doing ORM queries in parallel</li> <li>Allowing views to query external APIs without blocking threads</li> <li>Running slow-response/long-poll endpoints alongside each other efficiently</li> <li>Bringing easy performance improvements to any project that spends a majority of time blocking on databases or sockets (which is most projects!)</li> </ul></li> <li>Imperative that we keep Django backwards-compatible with existing code</li> <li>Why now? Django 2.1 will be the first release that only supports Python 3.5 and up, and so this provides us the perfect place to start working on async-native code</li> </ul> <p>Brian #5: <a href=""><strong>pydub</strong></a></p> <ul> <li>“Manipulate audio with a simple and easy high level interface”</li> <li>Really clean use of operators.</li> </ul> <pre><code> from pydub import AudioSegment # also handles lots of other formats song = AudioSegment.from_mp3("never_gonna_give_you_up.mp3") # pydub does things in milliseconds ten_seconds = 10 * 1000 first_10_seconds = song[:ten_seconds] last_5_seconds = song[-5000:] # boost volume by 6dB beginning = first_10_seconds + 6 # reduce volume by 3dB end = last_5_seconds - 3 # Concatenate audio (add one file to the end of another) without_the_middle = beginning + end </code></pre> <ul> <li>also: <ul> <li>crossfade</li> <li>repeat</li> <li>fade</li> <li>switch formats</li> <li>add metadata tags</li> <li>save with a specific bitrate</li> </ul></li> </ul> <p>Michael #6: <a href="">Molten: Modern API framework</a></p> <ul> <li>molten is a minimal, extensible, fast and productive framework for building HTTP APIs with Python.</li> <li>Heavy use of type annotations</li> <li>Officially supports Python 3.6 and later</li> <li>Request Validation: molten can automatically validate requests according to predefined schemas, ensuring that your handlers only ever run if given valid input</li> <li>Dependency Injection: Write clean, decoupled code by leveraging DI.</li> <li>Still experimental at this stage.</li> </ul>
Aug 07, 2018
#89 A tenacious episode that won't give up
<h1>Python Bytes 89</h1> <p>Sponsored by Datadog -- <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>tenacity</strong></a></p> <ul> <li>“Tenacity is a general-purpose retrying library to simplify the task of adding retry behavior to just about anything.”</li> <li>Example (Also, nice Trollhunters reference):</li> </ul> <pre><code> import random from tenacity import retry @retry def do_something_unreliable(): if random.randint(0, 10) &gt; 1: raise IOError("Broken sauce, everything is hosed!!!") else: return "Awesome sauce!" # Toby says this frequently print(do_something_unreliable()) </code></pre> <ul> <li>Features: <ul> <li>Generic Decorator API</li> <li>Specify stop condition (i.e. limit by number of attempts)</li> <li>Specify wait condition (i.e. exponential backoff sleeping between attempts)</li> <li>Customize retrying on Exceptions</li> <li>Customize retrying on expected returned result</li> <li>Retry on coroutines</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Why is Python so slow?</strong></a></p> <ul> <li>Answer this question: <em>When Python completes a comparable application 2–10x slower than another language, why is it slow and can’t we make it faster?</em> </li> <li>Here are the top theories: <ul> <li>“It’s the GIL (Global Interpreter Lock)”</li> <li>“It’s because its interpreted and not compiled”</li> <li>“It’s because its a dynamically typed language”</li> </ul></li> <li><strong>“It’s the GIL”</strong> <ul> <li>Modern computers come with CPU’s that have multiple cores</li> <li>For web apps, it might not matter (e.g. <a href=""></a> has 16 worker processes, <a href=""></a> has 8 workers)</li> </ul></li> <li><strong>“It’s because its an interpreted language”</strong> <ul> <li>I hear this a lot and I find it a gross-simplification of the way CPython actually works.</li> <li>JIT vs. NonJIT is interesting (startup time too)</li> </ul></li> <li><strong>“It’s because its a dynamically typed language”</strong> <ul> <li>In a “Statically-Typed” language, you have to specify the type of a variable when it is declared. Those would include C, C++, Java, C#, Go.</li> <li>In a dynamically-typed language, there are still the concept of types, but the type of a variable is dynamic.</li> <li>Not having to declare the type isn’t what makes Python slow</li> <li>It’s this design that makes it incredibly hard to optimize Python.</li> </ul></li> <li><strong>Conclusion</strong> <ul> <li>Python is primarily slow because of its dynamic nature and versatility. It can be used as a tool for all sorts of problems, where more optimized and faster alternatives are probably available.</li> </ul></li> </ul> <p><strong>Brian #3</strong>: <a href=""><strong>Keynoting with Mu</strong></a></p> <ul> <li>David Beazley gave his <a href="">EuroPython talk/demo “Die Threads”</a> using Mu.</li> <li>Article also notes that simple tools are great not just for learning, but for teaching, as the extra clutter of a full power editor doesn’t distract too much.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>A multi-core Python HTTP server (much) faster than Go (spoiler: Cython)</strong></a></p> <ul> <li>Exploring the question, “So, I’ve heard Python is slow… is it?”</li> <li>A multi-core Python HTTP server that is about 40% to 110% faster than Go can be built by relying on the Cython language and LWAN C library. </li> <li>Just a proof of concept validates the possibility of high performance system programming in the Cython language. </li> <li>Primarily interesting as a highlight of Cython <ul> <li>Cython is both an optimizing static compiler and a hybrid language. It mainly gives the ability to:</li> <li>write Python code that can call back and forth from and to C/C++;</li> <li>add static typing using C declarations to Python code in order to boost performance;</li> <li>release the GIL in some code sections.</li> </ul></li> <li>Cython generates very efficient C code, which is then compiled into a module that Python can import. So it is an ideal language for wrapping external C libraries, and for developing C modules that speed up the execution of Python code.</li> <li>However, all experiments we are aware that rely on Cython for system programming fail short in at least two ways: <ul> <li>as soon as some Python code is invoked (as opposed to pure Cython <code>cdef</code> code), <a href="">performance degrades by one or two orders of magnitude</a>;</li> <li>benchmarks are most of the time provided for single core execution only, which is somehow unfair considering Golang's ability to scale up on multiple cores.</li> </ul></li> </ul> <p><strong>Brian #5</strong>: <a href=""><strong>PyCharm 2018.2 beefs up pytest support</strong></a></p> <ul> <li>Honestly, I’m super excited about this release to help my team navigate to all of the fixtures I create on a regular basis.</li> <li>This is the release I’ve been waiting for.</li> <li>I can now fully utilize the power of pytest from PyCharm</li> <li>Here’s the few things that were missing that now work great: <ul> <li>Autocomplete fixtures from various sources</li> <li>Quick documentation and navigation to fixtures</li> <li>Renaming a fixture from either the definition or a usage</li> <li>Support for pytest’s parametrize</li> </ul></li> <li>See also: <a href="">PyCharm 2018.2 and pytest Fixtures</a></li> <li>But if you really want to understand fixtures quickly, <a href="">read chapters 3 and 4 of the pytest book.</a></li> </ul> <p><strong>Michael #6</strong>: <a href=""><strong>XAR for Facebook</strong></a></p> <ul> <li>XAR lets you package many files into a single self-contained executable file. This makes it easy to distribute and install.</li> <li>A <code>.xar</code> file is a read-only file system image which, when mounted, looks like a regular directory to user-space programs. This requires a one-time installation of a driver for this file system (<a href="">SquashFS</a>).</li> <li>There are two primary use cases for XAR files. <ul> <li>Simply collecting a number of files for automatic, atomic mounting somewhere on the filesystem. </li> <li>By making the XAR file executable and using the xarexec helper, a XAR becomes a self-contained package of executable code and its data. A popular example is Python application archives that include all Python source code files, as well as native shared libraries, configuration files, other data.</li> </ul></li> <li>Advantages of XAR for Python usage <ul> <li>SquashFS looks like regular files on disk to Python. This lets it use regular imports which are better supported by CPython.</li> <li>SquashFS looks like regular files to your application, too. You don't need to use <code>pkg_resources</code> or other tricks to access data files in your package.</li> <li>SquashFS with Zstandard compression saves disk space, also compared to a ZIP file.</li> <li>SquashFS doesn't require unpacking of <code>.so</code> files to a temporary location like ZIP files do.</li> <li>SquashFS is faster to start up than unpacking a ZIP file. You only need to mount the file system once. Subsequent calls to your application will reuse the existing mount.</li> <li>SquashFS only decompresses the pages that are used by the application, and decompressed pages are cached in the page cache.</li> <li>SquashFS is read-only so the integrity of your application is guaranteed compared to using virtualenvs or unpacking to a temporary directory.</li> </ul></li> <li>Performance is interesting too</li> </ul> <p>Extras:</p> <p>Brian:</p> <ul> <li><a href=""><strong>numpy 1.15.0</strong></a> just released recently. Switched testing to pytest.</li> </ul> <p>Michael:</p> <ul> <li>SciPy 2018 <a href=";index=1&amp;list=PLYx7XA2nY5Gd-tNhm79CNMe_qvi35PgUR"><strong>videos are out</strong></a></li> <li>PyOhio 2018 <a href=""><strong>videos are out</strong></a></li> <li><a href=""><strong>Call for papers</strong></a> at PyCon Canada in Toronto</li> <li><a href=""><strong>PyBay 2018</strong></a> conference in a few weeks</li> <li>My latest course, <a href=""><strong>Building data-driven web apps with Pyramid and SQLAlchemy</strong></a>, is out!</li> </ul>
Aug 04, 2018
#88 Python has brought computer programming to a vast new audience
<p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Documenting Python Code: A Complete Guide</strong></a></p> <ul> <li>Article describes the why you should document, comments vs docstrings vs separate documentation.</li> <li>Let’s zoom in on comments, because I don’t think many people get how to use comments effectively.</li> <li>Commenting <ul> <li>comments are for you and other developers to help maintain the code. They can also help users understand your mental model and design. the source is often used as documentation if the other docs are lacking or confusing or incomplete.</li> <li>Comments start with <code>#</code> and are not accessible at runtime.</li> <li>Comment uses: <ul> <li>planning and reviewing</li> <li>explaining intent</li> <li>explaining complicated algorithms</li> <li>tagging TODO, BUG, or FIXME sections.</li> </ul></li> <li>Article includes some good tips: <ul> <li>keep comments as close to code it’s describing as possible.</li> <li>don’t try to format it with ascii alignment or whatever</li> <li>minimal, most of your code shouldn’t need comments.</li> <li>remove planning comments when they aren’t needed any more</li> </ul></li> </ul></li> <li>Docstrings: <ul> <li>available at runtime via <code>help()</code>, <code>thing.__doc__</code>, and through many code completion tools in IDEs</li> <li>Can be used at function, class, module, and package level.</li> <li>Should help the user as if they don’t have the source available to look at.</li> </ul></li> <li>Also covered: <ul> <li>Commenting with type hints</li> <li>How to use docstrings.</li> <li>Docstring standard practices and formatting.</li> </ul></li> <li>Necessary elements of documenting projects</li> <li>Using tools like Sphinx, MkDocs, etc.</li> </ul> <p><strong>Michael #2: <a href="">Security vulnerability alerts for Python at Github</a></strong></p> <ul> <li>Last year, GitHub released security alerts that track security vulnerabilities in Ruby and JavaScript packages. </li> <li>They have identified millions of vulnerabilities and have prompted many patches.</li> <li>As of this week, Python users can now access the dependency graph and receive security alerts whenever their repositories depend on packages with known security vulnerabilities.</li> <li>See it under insights &gt; dependency graph</li> <li>Using it: <ul> <li>Ensure that you have checked in a requirements.txt or Pipfile.lock file inside of repositories that have Python code.</li> <li>Give access to private repos</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>How virtual environment libraries work in Python</strong></a></p> <ul> <li>“Have you ever wondered what happens when you activate a virtual environment and how it works internally? Here is a quick overview of internals behind popular virtual environments, e.g., virtualenv, virtualenvwrapper, conda, pipenv.”</li> <li>“When Python starts its interpreter, it searches for the site-specific directory where all packages are stored. The search starts at the parent directory of a Python executable location and continues by backtracking the path (i.e., looking at the parent directories) until it reaches the root directory. To determine if it's a site-specific directory, Python looks for the <code></code> module, which is a mandatory requirement by Python in order to work.”</li> <li>virtualenv creates a directory with some bin files, and the lib that mostly points to the parent Python site versions using symbolic links.</li> <li>Python 3.3, with PEP 405, added a pyvenv.cfg file that allows the interpreter itself to be a symbolic link, as well as an option to use system site packages, saving on lots of symbolic links at the start.</li> </ul> <p><strong>Michael</strong> 4:** <a href=""><strong>Qt for Python available at PyPi</strong></a></p> <ul> <li>Announcement: Finally the technical preview of Qt for Python is available at the Python Package Index (PyPI).</li> <li><code>pip install PySide2</code></li> <li>Try it at one of the demo apps <strong><a href=""></a></strong></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Learning (not) to Handle Exceptions</strong></a></p> <ul> <li>Understanding exceptions is important even if you never throw your own, since much of Python and 3rd party packages utilize them quite a bit.</li> <li>Try to catch specific exceptions. Don’t have <code>except:</code> catch everything.</li> <li>If you really need to intercept any exception, consider re-raising it with <code>raise</code></li> <li>Some tips with handling multiple exceptions.</li> <li><code>finally</code> can be used for stuff that needs to run regardless of an exception or not</li> <li><code>else</code> runs if no exception occurs.</li> <li>You can use both <code>finally</code> and <code>else</code></li> <li>Also: <ul> <li>tracebacks</li> <li>custom exceptions</li> <li>best practices</li> <li>adding arguments to exceptions</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Python has brought computer programming to a vast new audience</strong></a></p> <ul> <li>Features quotes from Guido van Rossum</li> <li>Interesting history</li> <li>Seeing with “outside eyes” is pretty novel and something we don’t often get to do.</li> <li>More about the meteoric growth of Python</li> <li>Warnings about AI in the hands of half educated novices</li> </ul>
Jul 27, 2018
#87 Guido van Rossum steps down
<p>Sponsored by Datadog: <a href=""><strong></strong></a> </p> <p>Special guests:</p> <ul> <li>Brett Cannon: <a href=""><strong>@brettsky</strong></a></li> <li>Carol Willing: <a href=""><strong>@WillingCarol</strong></a></li> </ul> <p>The topic: Guido steps down.</p> <p><strong>The announcement</strong>: <a href=""><strong>Transfer of Power</strong></a></p> <blockquote> <p><strong><em>Now that PEP 572 is done, I don't ever want to have to fight so hard for a PEP and find that so many people despise my decisions.</em></strong></p> <p><strong><em>I would like to remove myself entirely from the decision process. I'll still be there for a while as an ordinary core dev, and I'll still be available to mentor people -- possibly more available. But I'm basically giving myself a permanent vacation from being BDFL, and you all will be on your own.</em></strong></p> <p><em>After all that's eventually going to happen regardless -- there's still that bus lurking around the corner, and I'm not getting younger... (I'll spare you the list of medical issues.)</em></p> <p><strong><em>I am not going to appoint a successor.</em></strong></p> <p>So what are you all going to do? Create a democracy? Anarchy? A dictatorship? A federation?</p> <p><strong>I'm not worried about the day to day decisions</strong> in the issue tracker or on GitHub. Very rarely I get asked for an opinion, and usually it's not actually important. So this can just be dealt with as it has always been.</p> <p><strong><em>The decisions that most matter are probably</em></strong> <strong><em>- How are PEPs decided</em></strong> <strong><em>- How are new core devs inducted</em></strong></p> <p><em>We may be able to write up processes for these things as PEPs (maybe those PEPs will form a kind of constitution). But here's the catch. I'm going to try and let you all (the current committers) figure it out for yourselves.</em></p> <p><em>Note that there's still the CoC -- if you don't like that document your only option might be to leave this group voluntarily. Perhaps there are issues to decide like when should someone be kicked out (this could be banning people from python-dev or python-ideas too, since those are also covered by the CoC).</em></p> <p><em>Finally. A reminder that the archives of this list are public ( <a href=""></a> ) although membership is closed (limited to core devs).</em></p> <p><strong><em>I'll still be here, but I'm trying to let you all figure something out for yourselves. I'm tired, and need a very long break.</em></strong></p> <p>--Guido van Rossum (</p> </blockquote> <p><strong>Why it happened?</strong></p> <ul> <li>e.g. PEP 572 burn-out/treatment</li> <li>View the twitter thread on <a href=""><strong>this announcement tweet</strong></a></li> </ul> <p><strong>What this means?</strong></p> <ul> <li>“keep calm and keep coding”</li> </ul> <p><strong>Is there a danger of Python losing its momentum from this?</strong></p> <p><strong>What comes next?</strong></p> <ul> <li>current state of the governance discussion</li> </ul> <p><strong>What needs to be done to reduce this kind of pressure?</strong></p> <p>Brett’s <a href=""><strong>talk about setting open source expectations at PyCascades</strong></a> is very relevant.</p>
Jul 17, 2018
#86 Make your NoSQL async and await-able with uMongo
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>Special guest Bob Belderbos: <a href="">@bbelderbos</a></p> <p><strong>Brian #1:</strong> <a href=""><strong>responses</strong></a></p> <ul> <li>“A utility for mocking out the Python Requests library.”</li> <li>From Sentry</li> </ul> <p>Example:</p> <pre><code>import responses import requests @responses.activate def test_simple(): responses.add(responses.GET, '', json={'error': 'not found'}, status=404) resp = requests.get('') assert resp.json() == {"error": "not found"} assert len(responses.calls) == 1 assert responses.calls[0].request.url == '' assert responses.calls[0].response.text == '{"error": "not found"}' </code></pre> <p><strong>Bob #2:</strong> <a href=""><strong>29 common beginner Python errors on one page</strong></a></p> <ul> <li>Decision trees / graphics are nice to digest and concise, it wraps a lot of experience on one slide</li> <li>Knowing about common errors can safe you a lot of time (the guide I wish I had when I started coding in Python)</li> <li>Reminded me of struggles I had when I started in Python, for example TypeErrors when converting suspected ints to strings, regexes before discovering raw strings</li> <li>It made me think of related issues newer Pythonistas face, for example “I am reading a file but getting no input” can be translated to “I am looping over a generator for the second time and don’t get any output”</li> <li>Made me realize that some things are subtle, like comparing 3 == “3” or require good knowledge of stdlib (sorted returning new sequence vs inplace sort() for example)</li> <li>Made me reflect on how much hand holding you would give your students when teaching. Part of the learning is in the struggle. </li> <li>About the source, I like seeing Python being taught in all different kind of domains, in this case biology.</li> </ul> <p><strong>Michael #3:</strong> <a href=""><strong>μMongo</strong></a></p> <ul> <li>μMongo is a Python MongoDB ODM. </li> <li>It inception comes from two needs: <ul> <li>the lack of async ODM</li> <li>the difficulty to do document (un)serialization with existing ODMs.</li> </ul></li> <li>a few design choices: <ul> <li>Stay close to the standards MongoDB driver to keep the same API when possible: use <code>find({"field": "value"})</code> like usual but retrieve your data nicely OO wrapped !</li> <li>Work with multiple drivers (PyMongo, TxMongo, motor_asyncio and mongomock for the moment)</li> <li>Tight integration with Marshmallow serialization library to easily dump and load your data with the outside world</li> <li>i18n integration to localize validation error messages</li> <li>Free software: MIT license</li> <li>Test with 90%+ coverage ;-)</li> </ul></li> <li>async / await support through Motor</li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>Basic Statistics in Python: Descriptive Statistics</strong></a></p> <ul> <li>Cool use of Python to teach basic statistics topics. </li> <li>Includes code snippets to explain different concepts like min, max, mean, median, mode, …</li> <li>However, after you understand the math, DON’T write your own functions. <ul> <li>use built in Python functions and t<a href="">he statistics library built in to Python</a> (or numpy if you are on older Python versions).</li> </ul></li> </ul> <p>Example from article:</p> <pre><code>sum_score = sum(scores) num_score = len(scores) avg_score = sum_score/num_score avg_score &gt;&gt;&gt; 87.8884184721394 </code></pre> <p>Using built in:</p> <pre><code>&gt;&gt;&gt; x = (2, 2, 3, 100) &gt;&gt;&gt; min(x), max(x) (2, 100) &gt;&gt;&gt; import statistics as s &gt;&gt;&gt; s.mean(x), s.median(x), s.mode(x) (26.75, 2.5, 2) &gt;&gt;&gt; s.pstdev(x), s.pvariance(x) (42.29287765097097, 1788.6875) &gt;&gt;&gt; s.stdev(x), s.variance(x) (48.835608593184, 2384.9166666666665) </code></pre> <p><strong>Bob #5:</strong> <a href=""><strong>Strings and Character Data in Python</strong></a> </p> <ul> <li>Everything you need to know to work with strings and more …</li> <li>Similar to <a href="">that great itertools article</a> you shared some weeks ago: exhaustive overview</li> <li>Nice re-usable code snippets and explanation of basic concepts, ideal for beginners but you likely will get something out of it, few useful bites: <ul> <li>Instead of <code>try int(…) except</code>, you can use <code>isdigit()</code> on a string</li> <li>You can use <code>isspace()</code> to see if all characters of a nonempty string are whitespace characters ( <code>' '</code>, tab <code>'\t'</code>, and newline <code>'\n'</code>)</li> <li>It’s easy to make a header in your Python scripts: </li> </ul></li> </ul> <pre><code>&gt;&gt;&gt;&gt; 'bar'.center(10, '-') '---bar----' </code></pre> <pre><code>- Replace up till n occurrences: </code></pre> <pre><code>&gt;&gt;&gt;&gt; 'foo bar foo baz foo qux'.replace('foo', 'grault', 2) 'grault bar grault baz foo qux' </code></pre> <pre><code>- Strip multiple characters from both ends of a string: </code></pre> <pre><code>&gt;&gt;&gt;&gt; ''.strip('w.moc') 'realpython' </code></pre> <pre><code>- Add leading padding to a string with `zfill`: </code></pre> <pre><code>&gt;&gt;&gt;&gt; '42'.zfill(5) '00042' </code></pre> <ul> <li>This also reminded me of Python’s polymorphism, for example str.find and str.index work on both strings as well as lists</li> </ul> <pre><code> &gt;&gt;&gt; 'foo bar foo baz foo qux'.index('baz') 12 &gt;&gt;&gt; 'foo bar foo baz foo qux'.split().index('baz') 3 &gt;&gt;&gt; 'foo bar foo baz foo qux'.count('foo') 3 &gt;&gt;&gt; 'foo bar foo baz foo qux'.split().count('foo') 3 </code></pre> <p><strong>Michael #6:</strong> <strong>PEP 572</strong>: <a href=""><strong>Assignment expressions accepted</strong></a></p> <ul> <li>Whoa, check out that twitter conversation</li> <li>Splits 2 statements into an expressions (so they can be part of list comprehensions, etc).</li> <li>Not sure I like it but here you go:</li> </ul> <p>Example:</p> <pre><code># Handle a matched regex if (match := is not None: ... </code></pre> <p>Contrast old and new:</p> <pre><code># old if self._is_special: ans = self._check_nans(context=context) if ans: return ans # new if self._is_special and (ans := self._check_nans(context=context)): return ans </code></pre> <p><strong>Our news:</strong></p> <ul> <li>Michael: New course coming! Data-driven web apps in Pyramid</li> <li>Bob: Be sure to visit <a href="">PyBites Code Challenges</a></li> <li>Brian: More <a href="">Test and Code</a> episodes coming!</li> </ul>
Jul 13, 2018
#85 Visually debugging your Jupyter notebook
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>the state of type hints in Python</strong></a></p> <ul> <li>“Therefore, type hints <strong>should be used whenever unit test are worth writing.”</strong></li> <li>Type hints, especially for function arguments and return values, help make your code easier to read, and therefore, easier to maintain.</li> <li>This includes refactoring, allowing IDEs to help with code completion, and allow linters to find problems.</li> <li>For CPython <ul> <li>No runtime type inference happens.</li> <li>No performance tuning allowed. </li> <li>Of course, third party packages are not forbidden to do so.</li> </ul></li> <li>Non-comment type annotations are available for functions in 3.0+</li> <li>Variable annotations for 3.6+</li> <li>In 3.7, you can postpone evaluation of annotations with: from <code>__future__</code> import annotations</li> <li>Interface stub files <code>.pyi</code> files, are allowed now, but this is extra work and code to maintain. <ul> <li>typeshed has types for standard library plus many popular libraries.</li> </ul></li> <li>How do deal with multiple types, duck typing, and more discussed.</li> <li>A discussion of type generation and checking tools available now, including mypy</li> <li>See also: <a href="">Stanford Seminar - Optional Static Typing for Python - Talk by Guido van Rossum</a> <ul> <li>Interesting discussion that starts with a bit of history of where mypy came from.</li> </ul></li> </ul> <p><strong>Michael</strong> <strong>#2:</strong> <a href=""><strong>Django MongoDB connector</strong></a></p> <ul> <li>Via Robin on Twitter</li> <li>Use MongoDB as the backend for your Django project, without changing the Django ORM.</li> <li>Use Django Admin to access MongoDB</li> <li>Use Django with MongoDB data fields: Use MongoDB embedded documents and embedded arrays in Django Models.</li> <li>Connect 3rd party apps with MongoDB: Apps like Django Rest Framework and Viewflow app that use Django Models integrate easily with MongoDB.</li> <li>Requirements: <ul> <li>Python 3.6 or higher.</li> <li>MongoDB 3.4 or higher.</li> </ul></li> <li>Example</li> </ul> <div class="codehilite"><pre><span></span><code><span class="n">inner_qs</span> <span class="o">=</span> <span class="n">Blog</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">name__contains</span><span class="o">=</span><span class="s1">&#39;Ch&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">(</span><span class="s1">&#39;name&#39;</span><span class="p">)</span> <span class="n">entries</span> <span class="o">=</span> <span class="n">Entry</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">blog__name__in</span><span class="o">=</span><span class="n">inner_qs</span><span class="p">)</span> </code></pre></div> <p><strong>Brian #<em>*</strong>3<strong></em><em>:</strong> <strong></em><em><a href=""></strong>Python Idioms: Multiline Strings</em>*</a></p> <ul> <li>or “How I use dedent”</li> <li>Example:</li> </ul> <div class="codehilite"><pre><span></span><code> <span class="k">def</span> <span class="nf">create_snippet</span><span class="p">():</span> <span class="n">code_snippet</span> <span class="o">=</span> <span class="n">textwrap</span><span class="o">.</span><span class="n">dedent</span><span class="p">(</span><span class="s2">&quot;&quot;&quot;</span><span class="se">\</span> <span class="s2"> int main(int argc, char* argv[]) {</span> <span class="s2"> return 0;</span> <span class="s2"> }</span> <span class="s2"> &quot;&quot;&quot;</span><span class="p">)</span> <span class="n">do_something</span><span class="p">(</span><span class="n">code_snippet</span><span class="p">)</span> </code></pre></div> <p><strong>Michael #4:</strong> <a href=""><strong>Flaskerizer</strong></a></p> <ul> <li>A program that automatically creates Flask apps from Bootstrap templates </li> <li>Bootstrap templates from websites like <a href=""></a> and <a href=""></a> are a fast way to get very dynamic website up and running</li> <li>Bootstap templates typically don't work "out of the box" with the python web framework Flask and require some tedious directory building and broken link fixing before being functional with Flask. </li> <li>The Flaskerizer automates the necessary directory building and link creation needed to make Bootstrap templates work "out of the box" with Flask. </li> <li>Queue black turtleneck!</li> </ul> <p><strong>Brian #*5:</strong> <a href=""><strong>Learn Python the Methodical Way</strong></a></p> <ul> <li>From the article: <ul> <li>Make your way through a tutorial/chapter that teaches you some discrete, four-to-six-step skill.</li> <li>Write down those steps as succinctly and generically as possible.</li> <li>Put the tutorial/chapter and its solutions away.</li> <li>Build your project from scratch, peeking only when you’re stuck.</li> <li>Erase what you built.</li> <li>Do the project again.</li> <li>Drink some water.</li> <li>Erase what you built and do it again.</li> <li>A day or two later, delete your work and do it again – this time without peeking even once.</li> <li>Erase your work and do it again.</li> </ul></li> <li>The notion of treating code like you treat creative writing with rough drafts and sometimes complete do-overs is super liberating. </li> <li>You’ll be surprised how fast you can do something the second time, the third time, the fourth time. And it’s very gratifying.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>PixieDebugger</strong></a></p> <ul> <li>The Visual Python Debugger for Jupyter Notebooks You’ve Always Wanted</li> <li>Jupyter already supports pdb for simple debugging, where you can manually and sequentially enter commands to do things like inspect variables, set breakpoints, etc.</li> <li>Check out the video to get a good idea of its usage: <a href=""></a></li> </ul>
Jul 03, 2018
#84 Vibora web framework: It's fast, async, and means viper
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>Special guest <a href="">Nina Zakharenko</a> (<a href="">@nnja</a>) is a Cloud Developer Advocate at Microsoft!</p> <p><strong>Brian #1:</strong> <a href=""><strong>Correcting Documentation for a Deployed Python Package</strong></a></p> <ul> <li>"A clever way to release new documentation without releasing a new package that might confuse your user base.”</li> <li>Upload changes to pypi without bumping the version by using post release version numbers: <code>0.3.2</code> =&gt; <code>0.3.2.post1</code></li> <li>Prevent documentation issues by using <code>restview --long-description</code> before uploading. (or use md and really any md converter)</li> </ul> <p>Also:</p> <ul> <li><a href="">Packaging Python Projects</a> : revamped pypa tutorial that works pretty darned well.</li> <li><a href="">Using TestPyPI</a> : more detailed instructions on testing with TestPyPI before pushing to final spot.</li> </ul> <p><strong>Nina #2:</strong> <a href=""><strong>Flask Mega Tutorial</strong></a></p> <ul> <li>Amazing resource for developers who’d like to learn about building web applications with Flask in Python.</li> <li>Covers important topics like databases, internationalization, and dates and times. </li> <li>Three full sections on deploying your web app using Linux, Heroku, or containers. </li> <li><a href="">VS Code IDE has great Flask support</a>.</li> <li>Try Azure with a <a href="">$200 credit to deploy Flask apps</a>.</li> </ul> <p><strong>Michael</strong> <strong>#3:</strong> <a href=""><strong>10 common security gotchas in Python and how to avoid them</strong></a></p> <ul> <li>Article by Anthony Shaw (congrats on being a 2018 PSF Fellow)</li> <li>The 10 topics <ul> <li>Input injection (see <a href="">little bobby tables</a>)</li> <li>Use an ORM (db) or shlex module to escape input correctly (process)</li> <li>Parsing XML</li> <li>Assert statements</li> <li>Timing attacks</li> <li>A polluted site-packages or import path</li> <li>Temporary files</li> <li>Using yaml.load</li> <li>Pickles</li> <li>Using the system Python runtime and not patching it</li> <li>Not patching your dependencies</li> </ul></li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>pre-commit</strong></a> “A framework for managing and maintaining multi-language pre-commit hooks.”</p> <ul> <li>Describe pre-commit actions using yaml.</li> <li>Lots of projects already use it, like black. </li> <li>Does the work for you so you don’t have to read up on git commit hooks and such.</li> <li>Test out hooks ahead of time with <code>pre-commit run [HTML_REMOVED]</code></li> </ul> <p><strong>Nina #5: Python 3.7 release and PSF board members</strong></p> <ul> <li>Python 3.7 has just been released today! 🎉</li> <li><a href="">New Features Overview Blog Post</a></li> <li>Debugging improvements - new <code>breakpoint()</code> built-in function allows you to start an interactive session, like IPython. </li> <li>4 New PSF Board members elected - Congratulations to them! <ul> <li>Anna Ossowski</li> <li>Christopher Neugebauer</li> <li>Jeff Triplett</li> <li>Katie McLaughlin</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Vibora web framework</strong></a></p> <ul> <li>A new speedy web framework</li> <li>Only 14 days old, but has 21 contributors and 2k stars</li> <li>Just like Flask: Vibora APIs were heavily inspired by the awesome Flask.</li> <li>Schemas validation, template engine, sessions and many more features were written from scratch to provide great performance along with an elegant async interface.</li> <li>Vibora also take advantage of multiple CPU cores by default thanks to the multi-processed architecture. Uvloop and other C speed-ups are used when available.</li> <li>Virtual Hosts: Maybe you have different domains and you want to host them all with a single Vibora application. </li> <li>Deployment has its own HTTP app server</li> <li>Docs need help</li> </ul> <p>Our news and extras:</p> <ul> <li><a href="">Qt for Python Webinar</a> <ul> <li>via <a href="">Fredrik Averpil</a> <a href=""></a></li> </ul></li> </ul>
Jun 28, 2018
#83 from __future__ import braces
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p>Special guest: <a href="">Cristian Medina</a>, <a href="">@tryexceptpass</a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Code with Mu: a simple Python editor for beginner programmers.</strong></a></p> <ul> <li>Found out about this from Nicholas Tollervey (<a href="">@ntoll</a>)</li> <li>Built by an impressive list of people: <a href=""></a> thanks</li> <li>Beginning code editor that also works with Adafruit and micro:bit boards.</li> <li>From about: <ul> <li><strong>Less is More.</strong> <ul> <li>Mu has only the most essential features, so users are not intimidated by a baffling interface.</li> </ul></li> <li><strong>Tread the Path of Least Resistance.</strong> <ul> <li>Whatever the task, there is always only one obvious way to do it with Mu.</li> </ul></li> <li><strong>Keep it Simple.</strong> <ul> <li>It's quick and easy to learn Mu ~ complexity impedes a novice programmer's first steps.</li> </ul></li> <li><strong>Have fun!</strong> <ul> <li>Learning should inspire fun ~ Mu helps learners quickly create and test working code.</li> </ul></li> </ul></li> </ul> <p><strong>Cris #2:</strong> <a href=""><strong>Python parenthesis primer</strong></a></p> <ul> <li>Good for beginners. Covers the main uses of parenthesis, curly brackets and square brackets. Including code examples.</li> <li>Parenthesis <ul> <li>Callables.</li> <li>Operation prioritization.</li> <li>Tuples.</li> <li>Generator expressions.</li> <li>Skirting the indentation rules.</li> </ul></li> <li>Square brackets <ul> <li>Lists and their comprehensions.</li> <li>Indexing.</li> <li>Slices.</li> <li>Comments also mention type hints.</li> </ul></li> <li>Curly braces <ul> <li>Dictionaries and comprehensions.</li> <li>Sets and comprehensions.</li> <li>F-strings.</li> <li>str.format.</li> </ul></li> <li>Try to import braces from <code>__future__</code>:</li> </ul> <pre><code> &gt;&gt;&gt; from __future__ import braces File "[HTML_REMOVED]", line 1 SyntaxError: not a chance </code></pre> <p><strong>Michael #3:</strong> <a href=""><strong>Python for Qt Released</strong></a></p> <ul> <li>The Qt Company happy to announce the first official release of Qt for Python (Pyside2).</li> <li>v5.11</li> <li>We hope we can receive plenty of feedback on what works and what does not. We want to patch early and often.</li> <li>Eventually the aim is to release Qt for Python 5.12 without the Tech Preview flag.</li> <li>Started two years ago with this <a href="!topic/pyside-dev/pqwzngAGLWE">announcement</a> from <a href="">Lars</a>.</li> <li>Get Qt for Python: The release supports Python 2.7, 3.5 &amp; 3.6 on the three main desktop platforms. The packages can be obtained from or using pip with</li> <li><code>pip install --index-url= pyside2</code></li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>Itertools in Python 3, By Example</strong></a> <a href=""></a></p> <ul> <li>by David Amos (<a href="">@somacdivad</a>)</li> <li>Iterators and generators are awesome.</li> <li>Nice discussion of lazy evaluation and iterator algebra.</li> <li>Naive approach using list can blow up in memory and time if you use huge datasets.</li> <li>Examples: <ul> <li>combinations, combinations_with_replacement, permutations</li> <li>count, repeat, cycle, accumulate</li> <li>product, tee, islice, chain</li> <li>filterfalse, takewhile, dropwhile</li> </ul></li> </ul> <p><strong>Cris #5:</strong> <a href=""><strong>Python Sets and Set Theory</strong></a></p> <ul> <li>Nice primer on sets in python and a little set theory.</li> <li>How to build them: <code>set()</code> vs <code>{``'``value1``'``,</code> <code>'``value2``'``}</code> vs <code>{name for name in name_list}</code></li> <li>Membership tests (which are O(1))</li> <li>Set operations <ul> <li>Union</li> <li>Intersection</li> <li>Difference</li> <li>Symmetric Difference</li> </ul></li> <li>Frozen sets</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Python 3.7 is coming soon</strong></a><strong>!</strong></p> <ul> <li>Schedule <ul> <li>3.7.0 candidate 1: 2018-06-12</li> <li>3.7.0 final: 2018-06-27</li> </ul></li> <li>What’s new / changed? <ul> <li>New syntax features: PEP 563, postponed evaluation of type annotations.</li> <li>New modules: dataclasses: PEP 557 – Data Classes</li> <li>New built-in features: PEP 553, the new breakpoint() function.</li> <li>Standard lib changes: <ul> <li>The asyncio module has received new features, significant usability and performance improvements.</li> <li>The time module gained support for functions with nanosecond resolution.</li> </ul></li> <li>Speed: <ul> <li>Method calls are 20% faster</li> <li>3.7 is THE fastest Python available, period.</li> </ul></li> </ul></li> <li><a href=""><strong>What’s new in Python 3.7 course</strong></a> by Anthony Shaw</li> </ul> <p><strong>Our news</strong></p> <ul> <li>ahem… <strong><a href=""></a></strong></li> </ul>
Jun 22, 2018
#82 Let's make a clear Python 3 statement
<p>DigitalOcean: <a href=""><strong></strong></a></p> <p><em>* GitHub coverage coming at the end! *</em></p> <p><strong>Brian #1:</strong> <a href=""><strong>Building and Documenting Python REST APIs With Flask and Connexion</strong></a></p> <ul> <li><strong>Doug Farrell, <a href="">@writeson</a>, on the RealPython site.</strong></li> <li>Tutorial with example. <ul> <li>REST explanation of what REST is and is not</li> <li><a href="">Swagger</a>, swagger.yml to define API</li> <li>Use <a href="">Connexion</a> to incorporate swagger.yml into Flask app.</li> <li>Nice succinct explanation of swagger and API configuration.</li> <li>Demo of Swagger UI for API documentation.</li> <li>JavaScript included for MVC implementation.</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>MyPy + PyCharm</strong></a></p> <ul> <li>Written by Ivan Levkivskyi</li> <li>via Guido van Rossum</li> <li>Ricky Teachey asks: “What advantages does using mypy bring to pycharm vs just using pycharm's native type checking- which is already pretty good?”</li> <li>Response: <ul> <li>mypy is a bit more stricter/precise</li> <li>it is more configurable, lots of options regulating type system "rules"</li> <li>it typechecks the whole program, so that you immediately see errors your change causes in _other_ files</li> <li>people run mypy in CI and want to see the result before push</li> </ul></li> </ul> <p><strong>Brian #3: Automatic code/doc conversion</strong></p> <ul> <li><a href="">pyupgrade</a> <ul> <li>“A tool (and pre-commit hook) to automatically upgrade syntax for newer versions of the language.”</li> <li>Can even convert to f-strings with <code>--py36-plus</code> option.</li> </ul></li> <li><a href="">docs</a> <ul> <li>“Run <code>black</code> on python code blocks in documentation files.”</li> <li>blacken-docs provides a single executable (blacken-docs) which will modify .rst / .md files in place.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>python3statement</strong></a></p> <ul> <li>via Bruno Alla</li> <li>Matthias Bussonnier (Talk Python, <a href="">episode 44</a>) <ul> <li><em>“We now have 44 projects that pledged to drop #python2 in less than 30 months. Some already did ! To see which one, and how to migrate with as few disruption as possible for both Python 2 and 3 users head to ”</em></li> </ul></li> <li>Supporting legacy Python: **it is a small but constant friction in the development of a lot of code.</li> <li>We are keen to use Python 3 to its full potential, and we currently accept the cost of writing cross-compatible code to allow a smooth transition, but we don’t intend to maintain this compatibility indefinitely. </li> <li>Nice “Why switch to Python 3?” section and resources</li> <li>Nice list of participating projects <ul> <li>Can we get some that are not data science? :)</li> </ul></li> </ul> <p><strong>Microsoft buys GitHub</strong>:</p> <ul> <li><a href=";unread=1">Everyone complaining about Microsoft buying GitHub needs to offer a better solution</a></li> <li><a href="">Microsoft to acquire GitHub for $7.5 billion</a></li> <li><a href="">Linux Foundation: Microsoft's GitHub buy is a win for open source</a></li> <li>Coverage on Exponent podcast: <a href=" BihnqmtgQ">154 — Legacy Leverage</a></li> <li><a href=";utm_medium=comment_list">Nat Friedman, future CEO of GitHub, AMA</a></li> <li>Re gitlab: <ul> <li><a href="">GitLab congratulates GitHub and Microsoft</a></li> <li><a href="">GitLab imports from GitHub going up</a></li> </ul></li> </ul> <p>Our news and extras:</p> <ul> <li>PyLadies Cleveland just launched: <ul> <li>First meeting June 26</li> <li>(FB Profile) <a href=""></a></li> <li>(FB Community Page) <a href=""></a></li> <li>(Twitter) <a href=""></a></li> <li>(Meetup) <a href=""></a></li> <li>(YouTube) <a href=""></a></li> </ul></li> </ul>
Jun 15, 2018
#81 Making your C library callable from Python by wrapping it with Cython
<p>Sponsored by digitalocean: <a href=""></a></p> <p><strong>Brian #1: Learning about Machine Learning</strong></p> <ul> <li><a href=""><strong>hello tensorflow</strong></a> <ul> <li>one pager site with a demo of machine learning in action.</li> <li>“<strong>Machine Learning (ML)</strong> is the dope new thing that everyone's talking about, because it's really good at learning from data so that it can predict similar things in the future.”</li> <li>Includes a graphical demo of ML trying to learn the correct coefficients to a polynomial.</li> </ul></li> <li><a href="">Google Provides Free Machine Learning Course For All</a> <ul> <li><a href="">Machine Learning Crash Course</a> (MLCC) is a free 15 hours course that is divided into 25 lessons. It provides exercises, interactive visualizations, and instructional videos. These can help in learning machine learning concepts.</li> <li>40 exercises, 25 lessons, 15 hours, case studies, interactive visualizations</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Making your C library callable from Python by wrapping it with Cython</strong></a></p> <ul> <li>Article by Stav Shamir</li> <li>Cython is known for its ability to increase the performance of Python code. Another useful feature of Cython is making existing C functions callable from within (seemingly) pure Python modules.</li> <li>Need to directly interact from Python with a small C library</li> </ul> <p>Want to wrap this C function?</p> <pre><code>void hello(const char *name) { printf("hello %s\n", name); } </code></pre> <p>Just install Cython and write this:</p> <pre><code>cdef extern from "examples.h": void hello(const char *name) def py_hello(name: bytes) -&gt; None: hello(name) </code></pre> <p>Then create a setup file (details in article), call <code>python build_ext --inplace</code> and you’re good to go.</p> <p><strong>Brian #3:</strong> <a href=""><strong>Taming Irreversibility with Feature Flags (in Python)</strong></a></p> <ul> <li>“Feature Flags are a very simple technique to make features of your application quickly toggleable. The way it works is, everytime we change some behavior in our software, a logical branch is created and this new behavior is only accessible if some specific configuration variable is set or, in certain cases, if the application context respects some rules.”</li> </ul> <pre><code> def my_function(): if is_feature_active('feature_one'): do_something() else: do_something_else() </code></pre> <ul> <li>Benefits <ul> <li>Improving team’s response time to bugs. If a new feature causes a bunch of customer problems, just turn it off.</li> <li>Making possible to sync code more frequently. Merge to master with the feature turned off.</li> <li>Having a more fluid feature launching flow. Turn feature on in test/staging server.</li> <li>Validate your features easily with A/B testing, user groups, etc.</li> </ul></li> <li>Article discusses: <ul> <li>how to implement flags cleanly.</li> <li>measuring success with analytics</li> <li>implementing flags with third party packages and recommends a few.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>pretend: a stubbing library</strong></a></p> <ul> <li>Heard about this at the end of the pypi episode of Talk Python and wanted to highlight it more.</li> <li>Pretend is a library to make stubbing with Python easier.</li> <li>Stubbing is a technique for writing tests. A stub is an object that returns pre-canned responses, rather than doing any computation.</li> <li>Stubbing is related to mocking, but traditionally with stubs, you don’t care about behavior, you are just concerned with how your system under test responds to certain input data. <ul> <li>However, pretend does include a call recorder feature.</li> </ul></li> <li>Nice clean api:</li> </ul> <pre><code> &gt;&gt;&gt; from pretend import stub &gt;&gt;&gt; x = stub(country_code=lambda: "US") &gt;&gt;&gt; x.country_code() 'US' </code></pre> <pre><code> &gt;&gt;&gt; from pretend import stub, raiser &gt;&gt;&gt; x = stub(func=raiser(ValueError)) &gt;&gt;&gt; x.func() Traceback (most recent call last): File "[HTML_REMOVED]", line 1, in [HTML_REMOVED] File "", line 74, in inner raise exc ValueError </code></pre> <p><strong>Brian #5:</strong> <a href=""><strong>The official Flask tutorial</strong></a></p> <ul> <li>Has been updated recently. <ul> <li>simplified, updated, including the source code for the project.</li> <li>tutorial includes section on testing, including testing with pytest and coverage.</li> </ul></li> <li>Flask is part of <a href="">Pallets</a>, which develops and maintains several projects <ul> <li><a href="">Click</a> — A package for creating beautiful command line interfaces in a composable way</li> <li><a href="">Flask</a> — a flexible and popular web development framework</li> <li><a href="">ItsDangerous</a> — cryptographically sign your data and hand it over to someone else</li> <li><a href="">Jinja</a> — a full featured template engine for Python</li> <li><a href="">MarkupSafe</a> — a HTML-Markup safe string for Python</li> <li><a href="">Werkzeug</a> — a WSGI utility library for Python</li> </ul></li> <li>You can now donate to pallets to help with the maintenance costs of these important packages. <ul> <li>There’s a donate button on the pallets site that takes you to a PSF page. Therefore, donations are deductible in the US.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>An introduction to Python bytecode</strong></a></p> <ul> <li>Python is compiled</li> <li>Learn what Python bytecode is, how Python uses it to execute your code, and how knowing what it does can help you.</li> <li>Python is often described as an interpreted language—one in which your source code is translated into native CPU instructions as the program runs—but this is only partially correct. Python, like many interpreted languages, actually compiles source code to a set of instructions for a virtual machine, and the Python interpreter is an implementation of that virtual machine. This intermediate format is called "bytecode."</li> <li>These are your .PYC files</li> </ul> <p>Example:</p> <pre><code>def hello() print("Hello, World!") 2 0 LOAD_GLOBAL 0 (print) 2 LOAD_CONST 1 ('Hello, World!') 4 CALL_FUNCTION 1 </code></pre> <ul> <li>CPython uses a stack-based virtual machine. That is, it's oriented entirely around stack data structures (where you can "push" an item onto the "top" of the structure, or "pop" an item off the "top").</li> </ul> <p>View and explore using</p> <pre><code>import dis dis.dis(hello) </code></pre>
Jun 05, 2018
#80 Dan Bader drops by and we found 30 new Python projects
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Packaging Python Projects</strong></a></p> <ul> <li>Tutorial on the PyPA has been updated.</li> <li>Includes <code></code> instead of <code>REAMDE.rst</code></li> <li>Initial example of <code></code> no longer too minimal or too scary.</li> <li>Discussion of using <code>twine</code> to upload to before uploading to non-test pypi</li> <li><a href="">Related project, flit</a> <a href=""></a></li> </ul> <p><strong>Dan #2:</strong> <a href="">gidgethub — An async library for calling GitHub’s API</a></p> <ul> <li>Talk to GitHub API to add/modify issues, pull-requests, comments, …</li> <li>Also helpers to parse GitHub’s webhook events so you can write bots that react to new issues, comments, commits etc.</li> <li>Used it in @Mariatta’s GitHub Bot tutorial:<a href=""></a></li> <li>Cool architecture for a “modern” Python web API library (async, sansio, decorator based event callbacks) <ul> <li>supports different async backends: aiohttp, treq, Tornado <ul> <li>sans-I/O: “protocol implementations written in Python that perform <strong>no</strong> I/O (this means libraries that operate directly on text or bytes)”</li> <li>Why? → “<em>reusability</em>. By implementing network protocols without any I/O and instead operating on bytes or text alone, libraries allow for reuse by other code regardless of their I/O decisions. In other words by leaving I/O out of the picture a network protocol library allows itself to be used by both synchronous and asynchronous I/O code”</li> </ul></li> </ul></li> <li>(Biggest issue in that workshop was getting everyone upgraded to Python 3.6…but more on that later)</li> </ul> <p><strong>Michael #3:</strong> <a href=""><strong>pystemd</strong></a></p> <ul> <li>Recall I recently build a Python-based systemd service for geo syncing my course materials</li> <li>A thin Cython-based wrapper on top of libsystemd, focused on exposing the dbus API via sd-bus in an automated and easy to consume way.</li> <li>By Alvaro Leiva, a production engineer at Facebook / Instagram</li> <li><a href="">Presented at PyCon 2018</a></li> <li>Systemd: <ul> <li>Manages your services and their lifetimes</li> <li>e.g. I want my web app to start on boot but only after mongodb has started</li> </ul></li> <li><a href="">pystemd</a> lets you control and query these from a Python API</li> </ul> <p><strong>Brian #4:</strong> <a href=""><strong>PyCharm 2018.2 EAP 1 includes improved pytest support</strong></a></p> <ul> <li>From <a href="">Bruno Oliveira</a> <ul> <li>“Oh my, full support for <a href="">#</a><a href="">pytest</a> fixtures and parameterized tests coming in <a href="">@</a><a href="">pycharm</a> 2018.2.“</li> </ul></li> <li>“PyCharm 2018.2 supports <a href="">using fixtures in Pytest</a>. Using fixtures allows you to separate your setup code from the actual tests, making for more concise, and more readable tests. Additionally, there have been improvements to code navigation and refactoring Pytest tests, and to using parameterized tests.”</li> <li>It’s hard for me to fully express how FREAKING EXCITED I am about this.</li> <li>auto-complete now works with fixtures to test functions</li> <li>goto declaration now works with fixtures to test functions <ul> <li>(not fixtures of fixtures, but they know about that already)</li> </ul></li> <li>re-running a failed parametrization works (yay!)</li> <li>re-running a single parametrization works (yay!)</li> </ul> <p><strong>Dan #5:</strong></p> <ul> <li>Why is installing Python 3.6 so hard? (Recent GitHub Bot workshop experience)</li> <li>Sometimes hard to tell what’s easy/difficult for beginners</li> <li>People hit crazy edge cases: <ul> <li>running Linux Subsystem for Windows (WSL) on Windows host, install Python into wrong environment</li> <li>broken PPAs + bad StackOverflow advice → broken SSL and no pip on Ubuntu (deadsnakes PPA is the way to go)</li> <li>People install multiple Python environments: Anaconda + distribution</li> <li>Hard to find instructions for compiling from source on Linux</li> </ul></li> <li>Shameless plug: <a href=""><strong></strong></a></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>30 amazing Python projects (2018 edition)</strong></a> <a href=""></a></p> <ul> <li>Mybridge AI evaluates the quality by considering popularity, engagement and recency. To give you an idea about the quality, the average number of Github stars is 3,707.</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 30</strong></a><strong>:</strong> PDFTabExtract: A set of tools for extracting tables from PDF files helping to do data mining on scanned documents. </li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 28</strong></a><strong>:</strong> Surprise v1.0: A Python scikit for building and analyzing recommender systems </li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 27</strong></a><strong>:</strong> Eel: A little Python library for making simple Electron-like HTML/JS GUI apps </li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 25</strong></a><strong>:</strong> Clairvoyant: A Python program that identifies and monitors historical cues for short term stock movement — Have you seen <a href="">The Wall Street Code - VPRO documentary</a>?</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 21</strong></a><strong>:</strong> Fsociety: Hacking Tools Pack. A Penetration Testing Framework.</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 18</strong></a><strong>:</strong> Maya: Datetime for Humans in Python</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 16</strong></a><strong>:</strong> Better-exceptions: Pretty and useful exceptions in Python, automatically </li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 13</strong></a><strong>:</strong> Apistar: A fast and expressive API framework. For Python</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 8</strong></a><strong>:</strong> MicroPython: A lean and efficient Python implementation for microcontrollers and constrained systems</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 6</strong></a><strong>:</strong> spaCy (v2.0): Industrial-strength Natural Language Processing (NLP) with Python and Cython</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 2</strong></a><strong>:</strong> Pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration</li> <li><a href=";utm_medium=blog&amp;utm_campaign=read_more"><strong>No 1</strong></a><strong>:</strong> Home-assistant (v0.6+): Open-source home automation platform running on Python 3</li> </ul> <p><strong>Our news</strong></p> <ul> <li>Michael: Notable mention Cris’s GDPR writeup: <a href=""></a> </li> <li>Did you know about dropbox smart sync? <a href=""></a></li> </ul>
May 29, 2018
#79 15 Tips to Enhance your Github Flow
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>pytest 3.6.0</strong></a></p> <ul> <li>Revamp the internals of the <code>pytest.mark</code> implementation with correct per node handling which fixes a number of long standing bugs caused by the old design. This introduces new <code>Node.iter_markers(name)</code> and <code>Node.get_closest_mark(name)</code> APIs. - Depricating <code>Node.get_marker(name)</code>. - <a href="">reasons for the revamp</a> - <a href="">updating existing code to use the new APIs</a> - Now when <code>@pytest.fixture</code> is applied more than once to the same function a <code>ValueError</code> is raised. This buggy behavior would cause surprising problems and if was working for a test suite it was mostly by accident.</li> <li>Support for Python 3.7’s builtin <code>breakpoint()</code> method, - see <a href="">Using the builtin breakpoint function</a> for details. - Provided by friend of the show Anthony Shaw</li> <li><code>monkeypatch</code> now supports a <code>context()</code> function which acts as a context manager which undoes all patching done within the <code>with</code> block.</li> <li>whitespace only diffs in failed assertions include escaped characters to be easier to read.</li> <li>plus more… see <a href="">changelog</a></li> </ul> <p><strong>Michael #2: Hello</strong> <a href=""><strong>Qt for Python</strong></a></p> <ul> <li>The first Qt for Python technology preview release is almost here, and for this reason we want to give a brief example on how it will open the doors to the Python world.</li> <li>The real question is: how to access the methods of a Qt class? To simplify the process, we kept Qt APIs. (basically change -> to . in the API)</li> <li>Can it be more pythonic? “We want to include more Python flavor features to Qt for Python in the near future, but at the moment we are focusing on the TP.”</li> <li>The wheels situation: we are planning a set of wheels for Linux, macOS and Windows for 64bit and a 32bit version only for windows. </li> <li>AFAIK, this is Pyside2 reborn</li> </ul> <p><strong>Brian #3:</strong> <a href="!msg/mongodb-user/UWIPZEAKYiw/C5twgBwrCwAJ"><strong>MongoDB 4.0.0-rc0 available</strong></a></p> <ul> <li>MongoDB 4.0.0-rc0, the first release candidate of MongoDB 4.0, is out and is ready for testing. <ul> <li>Multi-document ACID transactions</li> <li>Non-Blocking Secondary Reads</li> <li>lots of other goodies, see <a href="!msg/mongodb-user/UWIPZEAKYiw/C5twgBwrCwAJ">announcement</a></li> <li>Did we mention <a href="">Transactions</a>!</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Pipenv review, after using it in production</strong></a></p> <ul> <li>Nice summary: “The current state of python’s packaging is awful, I don’t think there’s anyone who could disagree with that. This problem is recognized and there are many attempts at trying to solve the mess. Pipenv was the first and it has gained a lot of traction, however it doesn’t sit well with everyone. And it’s also not suited for every project — like libraries.”</li> <li>The multiple environment problem: The tl;dr is — supporting multiple environments goes against Pipenv’s (therefore also Pipfile’s) philosophy of deterministic reproducible application environments. So if you want to use Pipenvfor a library, you’re out of luck. That means many projects just can not use Pipenv for their dependency managment.</li> <li>The good <ul> <li>Pipfile and Pipfile.lock really are superior to requirements.txt. By a ton.</li> <li>While I disliked it at first, having flake8 and security check builtin in a single tool is great</li> <li>Installing (exclusively) from a private respository works very well. Instead of replacing a dotfile somewhere in the system, specifying the repository in Pipfile is great</li> <li>Creating a new Pipfile is very easy</li> <li>No problems introducing Pipenv to it’s new users</li> <li>No problems installing from a mixture of indexes, git repositores…</li> <li>With --sequential it is actually deterministic, as advertised</li> <li>Virtualenv is much easier to get into and understand</li> <li>Dependencies can be installed into system (e.g. in Docker) — our case.</li> <li>At no point did anyone in the team even mentioned getting rid of Pipenv — which is a lot better than it sounds</li> </ul></li> <li>Related: <ul> <li><a href="">PyCon 2018 talk about the history and future of Python packaging, including pipenv</a>.</li> <li><a href="">Recent changes to the official wording around pipenv</a> (removes the statement that it’s the official way of managing <strong>application</strong> dependencies)</li> </ul></li> </ul> <p><strong>Brian</strong> <strong>#5:</strong> <a href=""><strong>15 Tips to Enhance your Github Flow</strong></a></p> <ul> <li>using github projects to prioritize issues and track progress</li> <li>using tags on issues</li> <li>templates</li> <li>using hub and git-extras on command line</li> <li>commit message standards</li> <li>scoped commits</li> <li>style standards with pre-commit hooks</li> <li>automated tests and checks on pull requests</li> <li>protect master branch</li> <li>requiring code reviews</li> <li>squash pull requests</li> <li>…. more great topics</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Pandas goes Python 3 only</strong></a></p> <ul> <li>Via Randy Olseon</li> <li>It's official: Starting January 1, 2019, pandas will drop support for #Python 2. This includes no backports of security or bug fixes. </li> <li>Basically following NumPy’s lead</li> <li>The final release before <strong>December 31, 2018</strong> will be the last release to support Python 2. The released package will continue to be available on PyPI and through conda.</li> <li>Starting <strong>January 1, 2019</strong>, all releases will be Python 3 only.</li> </ul> <p><strong>Our news</strong></p> <ul> <li>It’s GDPR eve, are you ready? </li> <li>Need a GDPR laugh? See <a href=""></a></li> <li>XKCD Python Environments: <a href=""></a></li> </ul>
May 25, 2018
#78 Setting Expectations for Open Source Participation
<p>Sponsored by Datadog: <a href=""></a> Special guest: Kojo Idrissa -- <a href=""></a></p> <p>Brian #1: <a href=""><strong>The Forgotten Optional <code>else</code> in Python Loops</strong></a></p> <ul> <li>“Both for and while loops in Python also take an optional else suite (like the if statement and the try statement do), which executes if the loop iteration completes normally. In other words, the else suite will be executed if we don’t exit the loop in any way other than its natural way. So, no break statements, no return statement, or no exceptions being raised inside the loop.”</li> <li>Why? So you don’t have to invent a flag to indicate something wasn’t found if you are using the loop to search for something. </li> </ul> <p>Kojo #2: <a href=""></a></p> <ul> <li><a href=""></a> <ul> <li>Find out what your dependencies are!</li> <li>Look into <a href=""></a></li> </ul></li> </ul> <p>Michael #3: <a href="">The other (great) benefit of Python type annotations</a></p> <ul> <li>We've had type annotations for awhile</li> <li>When and why is sometimes unclear <ul> <li>Lack of types an issue sometimes, especially annoying while learning new APIs or diving into a new large codebase, and made me completely reliant on documentation.</li> <li>Optional: <ul> <li>You can’t break the code by adding them</li> <li>They have no effect performance-wise</li> <li>You may add them only where you see fit</li> </ul></li> </ul></li> <li>Straightforward benefits <ul> <li>Employ static code analysis to catch type errors prior to runtime</li> <li>Cleaner code/the code is self-documenting: “don’t use a comment when you can use a function or a variable”, we can now say “don’t use comments to specify a type, when you can use type annotation”</li> </ul></li> <li>The other benefit (it's massive!): Code completion</li> </ul> <p>Brian #4: <a href=";;t=48m55s"><strong>Setting Expectations for Open Source Participation</strong></a></p> <ul> <li>Or <strong>Pay for Open Source with Kindness</strong></li> <li>Brett Cannon’s morning talk this last Sunday at PyCon 2018</li> <li>This talk (or a variation of it and it’s content) is essential material for anyone working with open source.</li> <li>Everything in open source has a cost whether it’s time, effort, or emotional output.</li> <li>Open source should be a series of unsolicited kindnesses.</li> <li>Be open, considerate, and respectful</li> <li>Remember most of this runs on volunteer time and that people have lives.</li> <li>Guidelines for communicating online: <ul> <li>Assume you are asking for a favor.</li> <li>Assume your boss will read what you say.</li> <li>Assume your family will read what you say.</li> </ul></li> </ul> <p>Kojo #5:</p> <ul> <li>Python Community Events <ul> <li>Michael and I (along with Trey Hunner) helped lead a New Attendee Orientation</li> <li>Join your local Python community</li> <li>Be kind to your fellow Pythonistas</li> </ul></li> </ul> <p>Michael #6: <a href="">ngrok</a></p> <ul> <li>ngrok exposes local servers behind NATs and firewalls to the public internet over secure tunnels.</li> <li>Public URLs for testing on mobile devices, testing your chatbot, SSH access to your Raspberry Pi, sharing your local dev work on full stack web apps.</li> <li>Just a commandline away</li> <li>My use case: Course app development</li> <li>Features: <ul> <li>Secure Tunnels</li> <li>Request Inspection</li> <li>Fast (HTTP 2)</li> </ul></li> </ul> <p>Extras and our news:</p> <p>Michael:</p> <ul> <li>Live recording video is out: <a href=""></a></li> <li>Now up to 8 video servers around the world, Japan, Sao Paulo, and Mumbai are the latest. Based on the systemd thing we discussed way back when (<a href="">episode 54</a>)</li> </ul>
May 18, 2018
#77 You Don't Have To Be a Workaholic To Win
<p>Sponsored by Datadog: <a href=""><strong>pythonbytes.</strong></a><a href=""><strong>fm</strong></a><a href=""><strong>/datadog</strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Why Senior Devs Write Dumb Code</strong></a></p> <ul> <li>“<em>Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” - Kent Beck</em></li> <li>Code that is clean, straightforward, obvious, and easy to read actually takes practice to achieve.</li> <li>Follow principles like YAGNI, Singe Responsibility, DRY, etc.</li> <li>Avoid clever one-liners, weird abstractions.</li> <li>Esoteric language features.</li> <li>Code needs to be readable and easily understood while under time and stress pressure.</li> </ul> <p><strong>Michael #2:</strong> <a href="">GeoAlchemy 2</a></p> <ul> <li>GeoAlchemy 2 provides extensions to SQLAlchemy for working with spatial databases.</li> <li>GeoAlchemy 2 focuses on PostGIS. Aims to be simpler than its predecessor, GeoAlchemy.</li> <li>Using it: <ul> <li>Connect (e.g. Postgres)</li> <li>Declare a Mapping</li> </ul></li> </ul> <pre><code> class Lake(Base): __tablename__ = 'lake' id = Column(Integer, primary_key=True) name = Column(String) geom = Column(Geometry('POLYGON')) </code></pre> <ul> <li>Create a table (via the engine)</li> <li>Create an Instance of the Mapped Class</li> <li>Inserts like standard SQLAlchmey</li> <li>Spatial Query</li> </ul> <pre><code> from sqlalchemy import func query = session.query(Lake).filter( func.ST_Contains(Lake.geom, 'POINT(4 1)')) query = session.query(, Lake.geom.ST_Buffer(2).ST_Area().label('bufferarea')) </code></pre> <p><strong>Brian</strong> <strong>#3:</strong> <a href=""><strong>QtPyConvert</strong></a></p> <ul> <li>An automatic Python Qt binding transpiler to the abstraction layer.</li> <li>QtPyConvert supports the following bindings out of the box: <ul> <li><a href="">PyQt4</a></li> <li><a href="">PySide</a></li> <li><a href="">PyQt5</a></li> <li><a href="">PySide2</a></li> </ul></li> <li>Conversions leave code comments in place, with the help of RedBaron</li> <li>Converts to <a href=""></a> <ul> <li>Minimal Python 2 &amp; 3 shim around all Qt bindings - PySide, PySide2, PyQt4 and PyQt5</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href="">You Don't Have To Be a Workaholic To Win: 13 Alternative Ways To Stand Out</a></p> <ul> <li>Do we have to kill ourselves to get ahead?</li> <li>Don’t busy-brag</li> <li>Max Q analogy</li> <li>The tips</li> <li>Creativity</li> <li>Stubbornness</li> <li>Curiosity</li> <li>Kindness</li> <li>Planning</li> <li>Improvisation</li> <li>Enthusiasm</li> <li>Communication</li> <li>Presence</li> <li>Collaboration</li> <li>Willingness</li> <li>Patience</li> <li>Institutional Knowledge</li> </ul> <p><strong>Brian</strong> <strong>#5:</strong> <a href=""><strong>RedBaron</strong></a></p> <ul> <li>RedBaron is a python library to make the process of writing code that modify source code as easy and as simple as possible. <ul> <li>writing custom refactoring, generic refactoring, tools, </li> </ul></li> <li>Used by QtPyConvert to achieve the conversion while leaving code comments in place</li> <li>Uses the full syntax tree, FST. Like an AST, but keeps all information, including comments and formatting.</li> <li>possible uses: <ul> <li>rename a variable in a source file... without clashing with things that are not a variable (example: stuff inside a string)</li> <li>inline a function/method</li> <li>extract a function/method from a series of line of code</li> <li>split a class into several classes</li> <li>split a file into several modules</li> <li>convert your whole code base from one ORM to another</li> <li>do custom refactoring operation not implemented by IDE</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href="">Project Beeware AppStore</a></p> <ul> <li>Project BeeWare has just released its first iPhone app made in Python using its Briefcase tool.</li> <li>Simple travel app for currency and tip calculations</li> <li>Briefcase: A distutils extension to assist in packaging Python projects as standalone applications. Briefcase is a tool for converting a Python project into a standalone native application. You can package projects for: <ul> <li>Mac</li> <li>Windows</li> <li>Linux</li> <li>iPhone/iPad</li> <li>Android</li> <li>AppleTV</li> <li>tvOS</li> </ul></li> <li>While there are other Python GUI toolkits aiming to enable Python developers to build and deploy iOS apps, like for instance the very nice <a href="">Pythonista app</a>, the <a href="">BeeWare</a> project is a bit different because it aims at cross-platform compatibility and native widgets with a set of different tools, like Briefcase and Toga.</li> </ul> <p>Extras: </p> <ul> <li>Michael: Extra ssh breach Did you see that? <a href=""></a></li> <li>PyCon videos already up at <a href=""></a></li> </ul>
May 12, 2018
#76 Goodbye zero-versioning
<p>Sponsored by Datadog: <a href=""><strong>pythonbytes.</strong></a><a href=""><strong>fm</strong></a><a href=""><strong>/datadog</strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Unlearning toxic behaviors in a code review culture</strong></a></p> <ul> <li>unhelpful behaviors: <ul> <li>passing off opinion as fact</li> <li>overwhelming with an avalanche of comments</li> <li>asking people to fix problems they didn’t cause “while they’re at it”.</li> <li>asking judgmental questions</li> <li>being sarcastic</li> <li>using emojis</li> <li>not replying to comments</li> <li>ignoring (not calling out) toxic behavior from high performers</li> </ul></li> <li>helpful: <ul> <li>use questions or recommendations to drive dialog</li> <li>collaborate, don’t back-seat drive</li> <li>respond to every comment</li> <li>know when to take a discussion offline</li> <li>use opportunities to teach, and don’t show off</li> <li>don’t show surprise of lack of knowledge by others</li> <li>automate what can be</li> <li>refuse to normalize toxic behavior</li> <li>managers: hire carefully, listen to your team, and enforce</li> <li>set the standard as your team is small and growing</li> <li>understand you might be part of the problem</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href="">Flask 1.0 Released</a></p> <ul> <li>Dropped support for Python 2.6 and 3.3.</li> <li>The CLI is more flexible. <code>FLASK_APP</code> can point to an app factory, optionally with arguments. It understands import names in more cases where filenames were previously used. It automatically detects common filenames, app names, and factory names. <code>FLASK_ENV</code> describes the environment the app is running in, like <code>development</code>, and replaces <code>FLASK_DEBUG</code> in most cases. <a href="">See the docs to learn more.</a></li> <li>If python-dotenv is installed, the <code>flask</code> CLI will load environment variables from <code>.flaskenv</code> and <code>.env</code> files rather than having to export them in each new terminal.</li> <li>The development server is multi-threaded by default to handle concurrent requests during development.</li> <li>flask.ext, which was previously deprecated, is completely removed. Import extensions by their actual package names.</li> <li>Accessing missing keys from <code>request.form</code> shows a more helpful error message in debug mode, addressing a very common source of confusion for developers.</li> <li>Error handlers are looked up by code then exception class, on the blueprint then application. This gives more predictable control over handlers, including being able to handle <code>HTTPException</code>.</li> <li>The behavior of <code>app.logger</code> has been greatly simplified and should be much easier to customize. The logger is always named <code></code>, it only adds a handler if none are registered, and it never removes existing handlers. <a href="">See the docs to learn more.</a></li> <li>The <code>test_client</code> gained a <code>json</code> argument for posting JSON data, and the <code>Response</code> object gained a <code>get_json</code> method to decode the data as JSON in tests.</li> <li>A new <code>test_cli_runner</code> is added for testing an app's CLI commands.</li> <li>Many documentation sections have been rewritten to improve clarity and relevance. This is an ongoing effort.</li> <li>The <a href="">tutorial</a> and corresponding <a href="">example</a> have been rewritten. They use a structured layout and go into more detail about each aspect in order to help new users avoid common issues and become comfortable with Flask. <ul> <li>There are many more changes throughout the framework. <a href="">Read the full</a> </li> </ul></li> </ul> <p><strong>Brian</strong> <strong>#3:</strong> <strong>So, I still don’t quite get pipenv, ….</strong></p> <ul> <li>Best discussion of why pipenv is useful for applications I’ve come across so far is <a href="">Pipenv: A Guide to the New Python Packaging Tool</a></li> <li>Starts with a discussion of situations where <code>pip</code>, <code>pip freeze</code>, and <code>requirements.txt</code> fall apart. <ul> <li><code>requirements.txt</code> often just have an applications direct dependencies, not sub-dependencies.</li> <li><code>pip freeze &gt; requirements.txt</code> will pin your versions to specific versions, but then you’ve got to keep track of dependencies and sub-dependencies.</li> <li><code>Pipfile</code> intends to replace <code>requirements.txt</code>, with a simple-ish human readable format. Also includes extra things like dev environment support.</li> <li><code>Pipfile.lock</code> intends to replace pinned <code>requirements.txt</code> files. Also includes hashes to validate versions haven’t been corrupted.</li> <li><code>pipenv</code> also includes cool tools like dependency graphing, checking for updates, etc.</li> </ul></li> <li><code>pipenv</code> should be used for applications, but not packages intended to be included in other applications. But you can use it during package development, just probably not include the Pipfile and Pipfile.lock in the repo or package distribution. - Brian’s comment</li> </ul> <p>Bonus extra: </p> <ul> <li><a href="">cookiecutter-pipenv: Cookiecutter Python Package Template with Pipenv</a> </li> </ul> <p><strong>Michael #4</strong><a href=""><strong>:</strong></a> <a href="">GraalVM: Run Programs Faster Anywhere</a></p> <ul> <li>Why? <ul> <li>Current production virtual machines (VMs) provide high performance execution of programs only for a specific language or a very small set of languages.</li> <li>Compilation, memory management, and tooling are maintained separately for different languages, violating the ‘don’t repeat yourself’ (DRY) principle.</li> <li>high performance VMs are heavyweight processes with high memory footprint and difficult to embed.</li> </ul></li> <li>Oracle Labs started a new research project for exploring a novel architecture for virtual machines. Our vision was to create a single VM that would provide high performance for all programming languages, therefore facilitating communication between programs.</li> <li>Released: <a href="">GraalVM</a>, a universal virtual machine designed for a polyglot world.</li> <li>GraalVM provides high performance for individual languages and interoperability with zero performance overhead for creating polyglot applications.</li> <li>GraalVM 1.0 allows you to run: <ul> <li>JVM-based languages like Java, Scala, Groovy, or Kotlin</li> <li>JavaScript (including Node.js)</li> <li>LLVM bitcode (created from programs written in e.g. C, C++, or Rust)</li> <li>Experimental versions of Ruby, R, and <strong>Python</strong></li> </ul></li> <li>Future: This first release is only the beginning. We are working on improving all aspects of GraalVM; in particular the support for Python</li> </ul> <p><strong>Brian</strong> <strong>#5:</strong> <a href=""><strong>Testing a Flask Application using pytest</strong></a></p> <ul> <li>Small demo project, and article, that teaches the use of pytest in Flask.</li> <li>unit testing and functional testing</li> <li>Article covers testing models, with an example of a new user.</li> <li>project also has examples of using a test client to check the login page, password authentication, and a lot more.</li> <li>Very cool project.</li> </ul> <p><strong>Bonus:</strong> <a href=""><strong>A cool new pytest plugin: pytest-caprng</strong></a></p> <ul> <li>Tests that use <code>random</code> or <code>np.random</code> may fail, but when you re-run them, they don’t fail, which makes them hard to debug.</li> <li>This plugin adds pytest flags <code>--caprng-global-stdlib</code> and <code>--caprng-global-np</code>, which saves the random state before each test so that if you re-run the test, the random-ness is not so random, and you can reproduce your failure.</li> <li>Also, thanks John for reminding me what “stochastic” means.</li> </ul> <p><strong>Michael #6:</strong> <a href="">How to have a great first PyCon</a></p> <ul> <li>Spending your time: which talks should I go to? The talks at PyCon are typically uploaded to YouTube within 24 hours after the talk ends. I am suggesting that you don’t need to worry about attending every talk.</li> <li>Open spaces: attend them and consider hosting your own! </li> <li>There are a few reasons I often pick open spaces over talks: <ul> <li>Often the open spaces are more niche and topical than the talks: there are some subjects that exist in open spaces every year but which I’ve never seen a talk on</li> <li>Open spaces are all about interaction and discussion whereas talks are a monologue that often evolves into subsequent dialogues</li> <li>Open spaces aren’t recorded whereas the talks are, meaning you can’t really catch up on them later</li> </ul></li> <li>Tips for starting conversation, breakfast and lunch time…</li> <li>The hallway track 👣 <ul> <li>Something you might consider doing while at PyCon is taking breaks in the hallway. </li> <li>In addition to joining or starting a table in the hallway, consider identifying groups that have <a href="">a PacMan opening</a> to join and make sure the groups you’re in are PacMan-friendly.</li> </ul></li> <li>Interacting online during PyCon 🐦 <ul> <li>I recommend getting a Twitter account to make it easier to passively keep up with folks from PyCon after the conference ends.</li> <li>Sometimes people on Twitter will ask if anyone would like to join them for dinner and you might decide to reply and say you’d like to join.</li> </ul></li> <li>Networking isn’t a dirty word: it means making friends 👥</li> <li>I hear two opposing concerns sometimes expressed about PyCon: <ul> <li>Isn’t everyone here to get a job or hire people?</li> <li>Is it acceptable to go to PyCon looking for a job?</li> <li>PyCon is a networking event. That doesn’t necessarily mean everyone is there to get a job, but it also definitely doesn’t mean it’s unacceptable to job-seek at Python.</li> </ul></li> <li>Other topics include <ul> <li>Volunteering</li> <li>Evening events: dinners and board games</li> <li>Give a lightning talk ⚡</li> <li>Take care of yourself</li> </ul></li> <li>Final tip from commentor: If you are on windows, it's helpful to install a virtual image of a linux like the current ubuntu on your laptop, because you could run into situations where you want to follow a talk / training which doesn't work on windows and then you're missing a great opportunity to learn.</li> </ul> <p><strong>Our news</strong></p> <ul> <li>Come see us at PyCon!!! We’ll have stickers! </li> <li>Brian’s talk is Friday at 5 something. </li> <li>We are doing a live Python Bytes open session, join “friends of the show” to get notified</li> <li>I’ll be at Microsoft BUILD too</li> <li>PyGotham 2018 Call for Proposals</li> <li><a href="http://PyCon.DE">http://PyCon.DE</a> (24-26 October 2018 in Karlsruhe, Germany) starting our CfP tomorrow until May 20. <a href=""></a></li> </ul>
May 04, 2018
#75 officially launches
<p>Sponsored by Datadog: <a href=""><strong>pythonbytes.</strong></a><a href=""><strong>fm</strong></a><a href=""><strong>/datadog</strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>numba</strong></a></p> <ul> <li>From the numba readme: <ul> <li>“The easiest way to install numba and get updates is by using the Anaconda Distribution: <a href=""></a>”</li> </ul></li> <li><a href=";utm_medium=email&amp;utm_campaign=weekly#The-need-for-speed-without-bothering-too-much:-An-introduction-to-numba">The need for speed without bothering too much: An introduction to numba</a></li> <li>Can get huge speed up for some computation heavy loops or algorithms.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>pip 10 is out</strong></a><strong>!</strong></p> <ul> <li>Time for: <code>python -m pip install --upgrade pip</code></li> <li>Features: <ul> <li>Python 2.6 is no longer supported - if you need pip on Python 2.6, you should stay on pip 9, which is the last version to support Python 2.6.</li> <li>Support for PEP 518, which allows projects to specify what packages they require in order to build from source. (PEP 518 support is currently limited, with full support coming in future versions - see the documentation for details).</li> <li>Significant improvements in Unicode handling for non-ASCII locales on Windows.</li> <li>A new "pip config" command.</li> <li>The default upgrade strategy has become "only-if-needed"</li> <li>Many bug fixes and minor improvements.</li> </ul></li> </ul> <p><strong>Brian</strong> <strong>#3:</strong> <a href=""><strong>Keyword (Named) Arguments in Python: How to Use Them</strong></a></p> <ul> <li>Using keyword arguments is often seen when there are many arguments to a function that have useful defaults, and you only want to override the default with some of the arguments.</li> <li>Example:</li> </ul> <div class="codehilite"><pre><span></span><code> <span class="o">&gt;&gt;&gt;</span> <span class="k">print</span><span class="p">(</span><span class="s1">&#39;comma&#39;</span><span class="p">,</span> <span class="s1">&#39;separated&#39;</span><span class="p">,</span> <span class="s1">&#39;words&#39;</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s1">&#39;, &#39;</span><span class="p">)</span> <span class="n">comma</span><span class="p">,</span> <span class="n">separated</span><span class="p">,</span> <span class="n">words</span> </code></pre></div> <ul> <li>You can take positional arguments and require some to be named with various uses of <code>*</code></li> </ul> <pre><code> def foo(*, bar, baz): print(f'{bar} {baz}') </code></pre> <ul> <li>Lots of other useful tricks in this article.</li> </ul> <p><strong>Michael #4</strong><a href=""><strong>:</strong></a> <a href=""><strong> officially launches</strong></a></p> <ul> <li>Legacy PyPI shutting down April 30</li> <li>Listen to talk python 159</li> <li>Starting April 16, the canonical Python Package Index is at <a href=""></a> and uses the new Warehouse codebase.</li> <li>Launched the new PyPI, redirecting browser traffic and API calls (including "pip install") from to the new site. The old codebase is still available at <a href=""></a> for now.</li> <li>Monday April 30 (2018-04-30): We plan to shut down legacy PyPI . The address will continue to redirect to Warehouse.</li> <li>If your site/service links to or uses <a href=""></a>, you should start using instead: <a href=""></a> </li> </ul> <p><strong>Brian</strong> <strong>#5:</strong> <a href=""><strong>Python Modules and Packages – An Introduction</strong></a></p> <ul> <li>In Python, it is, and understanding modules and packages is key to getting a good footing when learning Python. It’s also an area that trips up people when they start trying to create reusable code.</li> <li>How to create a Python <strong>module</strong></li> <li>Locations where the Python interpreter searches for a module</li> <li>How to obtain access to the objects defined in a module with the <code>import</code> statement</li> <li>How to create a module that is executable as a standalone script</li> <li>How to organize modules into <strong>packages</strong> and <strong>subpackages</strong></li> <li>How to control package initialization</li> </ul> <p><strong>Michael #6: Pandas only like modern Python</strong></p> <ul> <li>From December 31st, 2018, Pandas will drop support for Python 2.7. This includes no backports of security or bug fixes (unless someone volunteers to do those)</li> <li>The final release before December 31, 2018 will be the last release to support Python 2. The released package will continue to be available on PyPI and through conda.</li> <li>Starting January 1, 2019, all releases will be Python 3 only.</li> <li>The full <a href="">reddit discussion</a> is interesting.</li> </ul> <p><strong>Our news</strong></p> <ul> <li>Just launched: Python 3, an illustrated tour! <a href=""><strong></strong></a></li> </ul>
Apr 28, 2018
#74 Contributing to Open Source effectively
<p>Sponsored by Datadog: <a href=""><strong>pythonbytes.</strong></a><a href=""><strong>fm</strong></a><a href=""><strong>/datadog</strong></a></p> <p><strong>Special guest: Matt Harrison -</strong> <a href=""><strong>__mharrison__</strong></a></p> <p><strong>Brian #1: Contributing to Open Source effectively</strong> The mechanics and conventions on how to contribute to open source projects can be confusing. After seeing a <a href="">very well documented pull request</a> that started with <code>[WIP]</code> in the subject line when it was first submitted, I tried to find out more about the conventions and mechanics of it all. I’m still learning, but here are a couple of resources:</p> <ul> <li><a href="">How to write the perfect pull request</a> is more of a mindset of how to initiate and receive PRs <ul> <li>Approach to writing a Pull Request, including that [WIP] trick.</li> <li>Offering feedback</li> <li>Responding to feedback</li> </ul></li> <li><a href="">Forge Your Future with Open Source</a>, <a href=""><strong>@</strong></a><a href="">vmbrasseur</a> book on contributing to open source, includes: <ul> <li>Make a Contribution, which includes PRs</li> <li>Make a difference without making a pull request, which is suggests many ways to contribute to a project without contributing code, like reviewing others contributions, testing, triaging bugs, …</li> <li>Interacting with the community.</li> </ul></li> </ul> <p><strong>Matt #2:</strong> <strong>Jupyter, Mathematica, and the Future of the Research Paper</strong></p> <ul> <li>Paul Romer, economy professor at NYU</li> <li>As a longtime Linux user there was constantly the question of the “year of the Linux Desktop”. Maybe this is the year of the “Jupyter desktop” (also beta version of JupyterLab). Not just a tool for innovators or early adopters</li> <li>Refers to Article in Atlantic contrasting Mathematica and Jupyter: <em>open-source developers have flocked to Python because it happens to be the de facto standard for scientific computing. Programming-language communities, like any social network, thrive—or die—on the strength of these feedback loops</em>. <a href=""> </a></li> </ul> <p>Quotes: </p> <p><em>Jupyter is a new open-source alternative [to Mathmatica] that is well on the way to becoming a standard for exchanging research results.</em></p> <p><em>Python libraries let me replicate everything I wanted to do with Mathematica: Matplotlib for graphics, SymPy for symbolic math, NumPy and SciPy for numerical calculations, Pandas for data, and NLTK for natural language processing. Jupyter makes it easy to use Latex to display typeset math. With Matplotlib, Latex works even in the label text for graphs. (I have not yet tried the major update, JupyterLab, which is still in beta testing.)</em> <em>I’m more productive. I’m having fun.</em> <a href=""></a></p> <p><strong>Michael</strong> <strong>#3:</strong> <a href=""><strong>Python Developers Survey 2017 Results</strong></a></p> <ul> <li>At the very end of 2017, JetBrains &amp; The PSF teamed up to build a solid picture of the modern Python developer</li> <li>Here are some take-aways <ul> <li>Almost 4 out of 5 Python developers use it as their main language, while for 21% it’s only a secondary language.</li> <li>Data analysis is as popular as web development with Python: Web development is the only category with a large gap (54% vs 33%) separating those using Python as their main language vs as a supplementary language. For other types of development, the differences are far less significant.</li> <li>At 28% to 27% application, There are as many Python web developers as Python data scientists</li> <li>Python 3 vs Python 2: 75% to 25% and accelerating</li> <li>Top Cloud Platform(s) <ul> <li>67%: AWS</li> <li>29%: Google App Engine</li> <li>26%: Heroku</li> <li>23%: DigitalOcean</li> <li>16%: Microsoft Azure</li> </ul></li> <li>Team Size <ul> <li>74%: 2-7 people</li> <li>16%: 8-12 people</li> <li>5%: 13-20 people</li> <li>2%: 21-40 people</li> <li>2%: &gt; 40 people</li> </ul></li> <li>Operating Systems <ul> <li>49%: Windows</li> <li>19%: Linux</li> <li>15%: MacOS</li> </ul></li> </ul></li> </ul> <p><strong>Brian #4</strong><a href=""><strong>:</strong></a> <a href=""><strong><em>*</a><a href=""></strong>EdgeDB: A New Beginning</em>*</a> This is “news you can’t use” so far, because the product isn’t here yet. So why am I excited and interested in this:</p> <ul> <li>It’s from Elvis <a href=""><strong>@</strong></a><a href="">elprans</a> and Yury <a href=""><strong>@</strong></a><a href="">1st1</a>, who have brought us asyncio and uvloop</li> <li>It’s not just a relational DB, it’s a DB based on PostgreSQL but with an entire new way to specify schema and interact with it.</li> <li>Goal is to be fast, user friendly, and remove the need for ORMs</li> </ul> <p><strong>Matt</strong> <strong>#5:</strong> <a href=""><strong>Yellowbrick library</strong></a> </p> <ul> <li>Visualization is important, I’ve found bugs by plotting before. Also important in evaluation of machine learning projects</li> <li>This is a project that has been around for about two years. I’ve recently adopted it in place of some home grown libraries for some consulting projects and in my corporate training</li> <li>Yellowbrick offers visualization for: <ul> <li>Features</li> <li>Classification</li> <li>Regression</li> <li>Clustering</li> <li>Text</li> </ul></li> <li>Like sk-learn, uses a similar api (.fit, .transform, .poof (plot))</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Depression AI</strong></a></p> <ul> <li>Alexa skill for people suffering with depression. </li> <li>Alexa <a href=";ie=UTF8&amp;qid=1522775645&amp;sr=1-1&amp;keywords=depression ai">store listing</a></li> <li>Based on <a href="">Flask-Ask</a> <ul> <li>Discussed on <a href="">Talk Python 146</a></li> </ul></li> <li>Valley Hackathon 2018 winner</li> <li>71% of people who make their bed in the morning report feeling happy. This was the inspiration behind DepressionAI. </li> <li>The aim behind this skill is to encourage people to perform daily activities that become very difficult when one is depressed.</li> <li>The skill detects positive and negative moods. </li> <li>If the user is having a bad day, it asks them a series of questions about what they have done that day (e.g. "Have you gotten out of bed?") and if they haven't, it encourages them to do so.</li> <li>Features <ul> <li>Mood evaluation by a highly empathetic Alexa bot</li> <li>Suicidal intention detection and prevention attempt</li> <li>Location-based therapy reccomendations</li> <li>Suggestions for small activites to improve the user's mood</li> <li>Displays informative cards in the Alexa app</li> </ul></li> <li>Sample Phrases <ul> <li>“Alexa, check on me."</li> <li>"I feel down."</li> <li>"I haven't got out of bed today."</li> <li>"Help me feel better."</li> <li>"Help me find a therapist"</li> </ul></li> </ul>
Apr 19, 2018
#73 This podcast comes in any color you want, as long as it's black
<p>Sponsored by Datadog: <a href=""><strong>pythonbytes.</strong></a><a href=""><strong>fm</strong></a><a href=""><strong>/datadog</strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Set Theory and Python</strong></a></p> <ul> <li>“Let’s talk about sets, baby …” is what I have in my head while reading this.</li> <li>Great overview of set theory and how to use the set data type in Python.</li> <li>Covered: <ul> <li>Creating sets</li> <li>Checking for containment (in, not in)</li> <li>union : set of things in either set or in both</li> <li>intersection: set of things in 2 sets</li> <li>difference: set of things in one set but not the other</li> <li>symmetric difference: set of things in either set but not in both</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Trio: async programming for humans and snake people</strong></a></p> <ul> <li>The Trio project’s goal is to produce a production-quality, permissively licensed, async/await-native I/O library for Python. Like all async libraries, its main purpose is to help you write programs that do multiple things at the same time with parallelized I/O. </li> <li>Compared to other libraries, Trio attempts to distinguish itself with an obsessive focus on usability and correctness. </li> <li>Concurrency is complicated; we try to make it easy to get things right.</li> <li>Trio was built from the ground up to take advantage of the latest Python features</li> <li>Inspiration from <a href="">many sources</a>, in particular Dave Beazley’s <a href="">Curio</a></li> <li>Resulting design is radically simpler than older competitors like asyncio and Twisted, yet just as capable.</li> <li>We <em>do</em> encourage you do use it, but you should <a href="">read and subscribe to issue #1</a> to get warning and a chance to give feedback about any compatibility-breaking changes.</li> <li>Excellent scalability: trio can run 10,000+ tasks simultaneously without breaking a sweat, so long as their total CPU demands don’t exceed what a single core can provide.</li> <li>Supports Python 3.5+ and PyPy</li> <li>Uses</li> </ul> <pre><code>, 3) trio.sleep(1.5) # Sleep, non-blocking async with trio.open_nursery() as nursery: print("parent: spawning child...") nursery.start_soon(child_func1) print("parent: spawning child...") nursery.start_soon(child_func2) print("parent: waiting for children to finish...") # -- we exit the nursery block here -- print("parent: child_func1 and child_func2 done!") </code></pre> <ul> <li>trio provides a <a href="">rich set of tools for inspecting and debugging your programs</a>.</li> <li>Consider <a href="">trio-asyncio</a> for compatibility</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>black: The uncompromising Python code formatter</strong></a></p> <ul> <li><p>An amusing take on code formatting. From the readme:</p> <ul> <li>“<em>Black</em> is the uncompromising Python code formatter. By using it, you agree to cease control over minutiae of hand-formatting. In return, <em>Black</em> gives you speed, determinism, and freedom from <code>pycodestyle</code> nagging about formatting. You will save time and mental energy for more important matters.”</li> <li>“Blackened code looks the same regardless of the project you're reading. Formatting becomes transparent after a while and you can focus on the content instead.”</li> <li>“<em>Black</em> makes code review faster by producing the smallest diffs possible.”</li> </ul></li> <li><p>Datadog is a monitoring solution that provides deep visibility and tracks down issues quickly with distributed tracing for your Python apps.</p></li> <li>Within minutes, you'll be able to investigate bottlenecks in your code by exploring interactive flame graphs and rich dashboards.</li> <li>Visualize your Python performance today, get started with a free trial with Datadog and they'll send you a free T-shirt.</li> </ul> <p>See for yourself, visit </p> <p><strong>Michael #4:</strong> <a href=""><strong>gain: Web crawling framework based on asyncio</strong></a></p> <ul> <li>Web crawling framework for everyone. Written with asyncio, uvloop and aiohttp.</li> <li>Simple and mostly automated <ul> <li>Define class mapped to CSS selectors and data to save</li> <li>Concurrently level</li> <li>Start URL</li> <li>Page templates to match URLs</li> <li>Run</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Generic Function in Python with Singledispatch</strong></a></p> <ul> <li>“Imagine, you can write different implementations of a function of the same name in the same scope, depending on the types of arguments. Wouldn’t it be great? Of course, it would be. There is a term for this. It is called “Generic Function”. Python recently added support for generic function in Python 3.4 (<a href="">PEP 443</a>). They did this to the <code>functools</code> module by adding <code>@singledispatch</code> decorator.”</li> <li>For people less familiar with “generic functions”. I think of this as providing similar functionality as C++’s function overloading.</li> <li>Allows you do things like this (full code example is in the article):</li> </ul> <pre><code> from functools import singledispatch @singledispatch def fprint(data): "code for default functionality" @fprint.register(list) @fprint.register(set) @fprint.register(tuple) def _(data): "code for list, set, tuple" @fprint.register(dict) def _(data): "code for dict" </code></pre> <p>More complete code example: </p> <pre><code> from functools import singledispatch @singledispatch def fprint(data): print(f'({type(data).__name__}) {data}') @fprint.register(list) @fprint.register(set) @fprint.register(tuple) def _(data): formatted_header = f'{type(data).__name__} -&gt; index : value' print(formatted_header) print('-' * len(formatted_header)) for index, value in enumerate(data): print(f'{index} : ({type(value).__name__}) {value}') @fprint.register(dict) def _(data): formatted_header = f'{type(data).__name__} -&gt; key : value' print(formatted_header) print('-' * len(formatted_header)) for key, value in data.items(): print(f'({type(key).__name__}) {key}: ({type(value).__name__}) {value}') # &gt;&gt;&gt; fprint('hello') # (str) hello # &gt;&gt;&gt; fprint(21) # (int) 21 #... # &gt;&gt;&gt; fprint({'name': 'John Doe', 'age': 32, 'location': 'New York'}) # dict -&gt; key : value # ------------------- # (str) name: (str) John Doe # (str) age: (int) 32 # (str) location: (str) New York </code></pre> <p><strong>Michael #6:</strong> <a href=""><strong>Unsync: Unsynchronizing async/await in Python 3.6</strong></a></p> <ul> <li>A rant about async/await in Python (by Alex Sherman)</li> <li>What’s wrong? <ul> <li>The two big friction points I’ve had are: <ul> <li>Difficult to “fire and forget” async calls (need to specifically run the event loop)</li> <li>Can’t do blocking calls to asyncio.Future.result() (it throws an exception)</li> </ul></li> <li>We need to acquire an even loop, do some weird call to execute the async function in that event loop, and then synchronously execute the event loop ourselves. </li> </ul></li> <li>What can we do? <ul> <li>C# had this great idea of executing each Task (their version of a Future) first synchronously in the main thread until an await is hit, and then queueing it into an ambient thread pool to continue later possibly in a separate thread. </li> <li>Python did not take this approach and my hunch is that the Python maintainers didn’t want to add an ambient thread pool to their language (which makes sense). </li> <li>Alex, however, is not the Python maintainers and did add an ambient thread (singular). I stuffed all the boiler plate into a decorator and the result looks like this:</li> </ul></li> </ul> <pre><code> @unsync async def unsync_async(): await asyncio.sleep(0.1) return 'I like decorators' print(unsync_async().result()) </code></pre> <ul> <li>using @unsync on a regular function (not an async one) will cause it to be executed in a ThreadPoolExecutor. </li> <li>To support CPU bound workloads, you can use @unsync(cpu_bound=True) to decorate functions which will be executed in a ProcessPoolExecutor</li> </ul>
Apr 12, 2018
#72 New versioning: Episode (with 72 releases)
<p>Sponsored by Datadog: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>ZeroVer: 0-based Versioning</strong></a></p> <ul> <li>“Software's most popular versioning scheme!”</li> <li>“<em>Cutting-edge software versioning for minimalists”</em></li> <li>My favorite April Fools prank this year. <ul> <li>Calls out many popular projects for never reaching 1.0</li> </ul></li> <li>From the about page: <ul> <li>“ZeroVer is the world's most popular software versioning convention, and the only one shown to harness the innovative power of zero. The benefits are innumerable and the effects on the software world are profound.”</li> <li>“Version 0.0.1 of ZeroVer was published by <a href="">Mahmoud Hashemi</a>, with help from Moshe, Mark, Kurt, and other patient collaborators, on 2018-04-01. ZeroVer is satire, <a href="">please do not use it</a>. We sincerely hope no project release schedules were harmed as a result of this humble attempt at programmer humor.”</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>GitHub Security Alerts Detected over Four Million Vulnerabilities</strong></a></p> <ul> <li>Last year GitHub launched “GitHub security alerts”</li> <li>GitHub’s security alerts notify repository admins when library vulnerabilities from the Common Vulnerabilities and Exposures (CVEs) list are detected in their repositories. </li> <li>Nearly half of all displayed alerts are responded to within a week and the rate of vulnerabilities resolved in the first seven days has been about 30%. </li> <li>When that statistics is restricted to only repositories with recent contributions, i.e., contributions in the last 90 days, things look even brighter, GitHub says, with 98% of such repositories being patched in fewer than seven days. </li> <li>More than four million vulnerabilities in over 500,000 repositories have been reported.</li> <li>Security alerts are only currently supported for repositories written in Ruby or JavaScript, while support for Python is planned for 2018.</li> <li>I also recommend <a href=""></a></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Markdown Descriptions on PyPI</strong></a></p> <ul> <li>Dustin Ingram provides detailed steps on how to get this to work.</li> <li> now supported by <ul> <li>“Only <a href=""></a> will correctly render your new Markdown description.</li> <li>Legacy PyPI (<a href=""></a>) will still render your description as plaintext, but don’t worry, <a href="">it’s going away real soon</a>.</li> </ul></li> <li>And also, <a href="">Github-Flavored Markdown Descriptions are supported</a>. <ul> <li>Another post, this one by Jon Wayne Parrot</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Concurrency comparison between NGINX-unit and uWSGI</strong></a></p> <ul> <li>show performance of two web application servers <ul> <li>nginx-unit (a new modern application web server)</li> <li>uWSGI (the best one application server)</li> </ul></li> <li>uWSGI and nginx-unit configured with 4 workers because test system has 4 cores.</li> <li>Effectively an empty “Hello world” Flask app</li> <li>Have a look at the pictures here:</li> <li>Take away: I’m going to start paying attention to NGINX-unit</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Loop better: A deeper look at iteration in Python</strong></a></p> <ul> <li>via Trey Hunner</li> <li>Generators are a great way to loop, but have a few gotchas <ul> <li>Looping twice doesn’t work</li> <li>Containment checks muck up the generator “contents”.</li> <li>Unpacking has odd results.</li> </ul></li> <li>This article describes Python loops in detail and then applies that do describe why the gotchas act like they do.</li> <li>Covered: <ul> <li>iterators, iterables, sequences, generators</li> <li>the iterator protocol</li> </ul></li> <li>Reading this will make you a better programmer, but might hurt your head.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Misconfigured Django Apps Are Exposing Secret API Keys, Database Passwords</strong></a></p> <ul> <li>Security researchers have been stumbling upon misconfigured Django applications that are exposing sensitive information such as API keys, server passwords, or AWS access tokens.</li> <li>He discovered 28,165 Django apps just this week where admins left debug mode enabled.</li> <li>Just by skimming through a few of the servers, the researcher found that the debug mode of many of these apps were exposing extremely sensitive information that would have allowed a malicious actor full access to the app owner's data. </li> <li>This is <strong>not a failure from Django's side</strong>. My recommendation is to disable debugging mode when deploying the application to production.</li> <li>Security researcher Victor Gevers said some of the servers running Django apps have already been compromised.</li> <li>He found at least one compromised server, running the Weevely web shell. Some servers Gevers found leaking sensitive data belonged to various government agencies carrying out critical operations.</li> <li>Gevers said he started notifying servers owners about their leaky Django apps. "At this moment we have reported 1,822 servers," Gevers said. "143 were fixed or taken offline."</li> </ul> <p><strong>Extra:</strong></p> <ul> <li>We covered wagtail on <a href="">episode 70</a>. They are running <a href="">a kickstarter campaign</a> to get some new features out. There’s a video there.</li> </ul>
Apr 05, 2018
#71 We can migrate to Python 3, careful please
<h1>Python Bytes 71</h1> <p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p>Special guest: Trey Hunner (<a href=""><strong>@treyhunner</strong></a>)</p> <p><strong>Trey #1:</strong> <a href=""><strong>The Conservative Python 3 Porting Guide</strong></a></p> <ul> <li>by <a href="">various Red Hat folks</a> mostly</li> <li>Python 2 is coming to the end of its life on January 1, 2020. Are you ready?</li> <li>This is one of the best guides I’ve found to porting your code from Python 2 to Python 3</li> <li>One of the issues with many of the Python 3 porting guides is that the old ones recommend dropping Python 2 support suddenly, which isn’t recommended anymore.</li> <li>I do wish this guide recommended the future library instead of python-modernize. They’re both great, but modernize is a little less focused on writing things the Python 3 way and a little more focused on just getting your code working in both 2 and 3.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>World-Class Software Companies That Use Python</strong></a></p> <ul> <li>by <a href="">Jason Reynolds</a> </li> <li>While it’s easy to see how you can tinker with Python, you might be wondering how this translates to actual business and real world applications.</li> <li><strong>Industrial Light and Magic</strong> <ul> <li>The studio has used Python in multiple other facets of their work. Developers use Python to track and audit pipeline functionality, maintaining a database of every image produced for each film.</li> </ul></li> <li><strong>Google</strong> <ul> <li>In the beginning, the founders of Google made the decision of “Python where we can, C++ where we must.”</li> <li>Currently powers YouTube among other things</li> </ul></li> <li><strong>Facebook</strong> <ul> <li>Ensures that the infrastructure of Facebook is able to scale efficiently</li> </ul></li> <li><strong>Instagram</strong> <ul> <li>the Instagram engineering team boasted that they were <a href="">running the world’s largest deployment of the Django web framework, which is written entirely in Python</a>.</li> <li>Instagram’s engineering team has invested time and resources into keeping their Python deployment viable at the massive scale (<a href="">~800 million monthly active users</a>) they’re operating at.</li> <li><a href="">PyCon 2017 keynote talk</a> by Lisa Guo and Hui Ding</li> </ul></li> <li><strong>Spotify</strong> <ul> <li>This music streaming giant is a <a href="">huge proponent of Python</a>, using the language primarily for data analysis and back end services. </li> <li>On the back end, there are a large number of services that all communicate over 0MQ, or <a href="">ZeroMQ</a>, an open source networking library and framework that is written in Python and C++(among other languages).</li> </ul></li> <li><strong>Quora</strong> <ul> <li>choosing to use Python where they could because of its ease of writing and readability, and implemented C++ for the performance critical sections. </li> <li>They got around Python’s lack of typechecking by writing unit tests that accomplish much the same thing.</li> <li>Another key consideration for using Python was the existence of several good frameworks at the time including Django and Pylons. </li> </ul></li> <li><strong>Netflix</strong> <ul> <li>Lots of infrastructure and ops work done via Python</li> </ul></li> <li><strong>Dropbox</strong> <ul> <li>Dropbox makes heavy use of Python</li> <li>Guido van Rossum works there!</li> <li>Lots of open source projects</li> <li>Client app in Python too</li> </ul></li> <li><strong>Reddit</strong> <ul> <li>This website had 542 million visitors every month across 2017, making it the fourth most visited website in the United States and seventh most visited in the world. </li> <li>In 2015, there were 73.15 million submissions and 82.54 billion pageviews. </li> <li>Behind it all, forming the software backbone, was Python.</li> </ul></li> </ul> <p><strong>Trey #3:</strong> <a href=""><strong>Stop Writing Classes</strong></a></p> <ul> <li>by <a href="">Jack Diederich</a></li> <li>This is one of my favorite PyCon talks to recommend to folks switching to Python from other programming languages. I especially like to recommend this talk to folks moving to Python from Java and C++.</li> <li>This is kind of an old talk. It's from 2012, so it's from the days of Python 2 but everything in it is still <em>very applicable today</em>.</li> <li>One of the great things about this talk is it doesn’t just show times that you should write functions instead of classes, it also shows an example or two of when classes really make sense.</li> <li>The big advice from this talk: if you have a class that only has two methods and one is the initializer, you probably need a function instead.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong> is alive</strong></a></p> <ul> <li>For the LONGest time, pypi has been run out of <a href=""></a></li> <li>Now the new version of pypi is out at</li> <li>Rewritten in Pyramid</li> <li>Do you want to contribute? Now the barriers have come down</li> <li><strong><a href="">Tweet with graphs</a></strong></li> </ul> <p><strong>Trey</strong> <strong>#5:</strong> <a href=""><strong>Pragmatic Unicode</strong></a></p> <ul> <li>by <a href="">Ned Batchelder</a></li> <li>Another PyCon 2012 talk that is still relevant today, though it does use quite a bit of Python 2 syntax</li> <li>Ned describes the unicode sandwich in this talk. Talks with good metaphors really help shape your mental model of a topic. This was the talk that helped me really understand the unicode vs bytes issue that Python 3 largely solves for us (or at least forces us to do so upfront).</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>pygame on pypy usable</strong></a></p> <ul> <li>via René Dudfield</li> <li>0.5x to 30x the speed</li> <li>That is pygame (same one that runs on cpython), works on pypy through its C extension API</li> <li><strong>This is exciting because</strong>: <ul> <li>pure python code being fast on pypy(after warmup), also mixed with the fast bits in C/asm.</li> <li>cpyext is getting faster in pypy. There is already work and discussion towards it being faster than CPython.</li> <li>maintaining one pygame code base is easier than maintaining several (pygame cffi/ctypes/cython, ...).</li> <li>with one code base it should be fast on both pygame, and pypy(in time).</li> </ul></li> <li><strong>Where it can be slower</strong>: if you are going into C code for a lot of small operations. Like when using lots of pygame.Rect in a tight loop. This is because (currently) the cost of going from PyPy code into and out of CPython API code (like pygame) is a bit slow.</li> <li>Ray tracing in PyGame: <ul> <li>On PyPy - 18.6 seconds.</li> <li>On Python 2.7 - 9 minutes, 28.1 seconds (30x slower)</li> </ul></li> </ul> <p><strong>Follow up and other news</strong></p> <p><strong>Michael:</strong> <strong>#100DaysOfCode in Python course</strong>: <strong><a href=""></a></strong></p> <p><strong>Trey:</strong> Python Morsels: <strong><a href=""></a></strong></p>
Mar 28, 2018
#70 Have you seen my log? It's cute!
<p>Sponsored by DigitalOcean: <a href=""><strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Online CookieCutter Generator</strong></a></p> <ul> <li>“Get a ZIP-archive with project by filling out the form.”</li> <li>By <a href="">@kpavlovsky_pro</a> Konstantin Pavlovsky</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>cutelog – GUI for Python's logging module</strong></a></p> <ul> <li>This is a graphical log viewer for Python's standard logging module. </li> <li>Features <ul> <li>Allows any number of simultaneous connections</li> <li>Fully customizable look of log levels and columns</li> <li>Filtering based on level and name of the logger, as well as filtering by searching</li> <li>Search through all records or only through filtered ones</li> <li>View exception tracebacks or messages in a separate window</li> <li>Dark theme (with its own set of colors for levels)</li> <li>Pop tabs out of the window, merge records of multiple tabs into one</li> </ul></li> <li>Based on PyQt5 speaking of GUIs</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>wagtail 2.0</strong></a></p> <ul> <li>“Wagtail is a content management system built on Django. It’s focused on user experience, and offers precise control for designers and developers.”</li> <li><a href="">The Zen of Wagtail</a> - nice philosophy of the project page to let you know if this kind of thing is right for you and your project.</li> <li>In 2.0 <ul> <li>a new text editor</li> <li>Django 2 support </li> <li>better scheduled publishing</li> <li>…</li> </ul></li> <li><a href="">wagtail docs</a></li> <li><a href="">gallery of sites made with wagtail</a></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>peewee 3.0 is out</strong></a></p> <ul> <li>Peewee is a simple and small ORM. It has few (but expressive) concepts, making it easy to learn and intuitive to use. <ul> <li>A small, expressive ORM</li> <li>Written in python with support for versions 2.7+ and 3.4+ (developed with 3.6)</li> <li>Built-in support for SQLite, MySQL and Postgresql.</li> <li>Numerous extensions available (postgres hstore/json/arrays, sqlite full-text-search, schema migrations, and much more).</li> </ul></li> <li>Although this was pretty much a complete rewrite of the 2.x codebase, I have tried to maintain backwards-compatibility for the public APIs.</li> <li>Exciting because of its async support via <strong>peewee-async</strong> <ul> <li><strong>peewee-async</strong> is a library providing asynchronous interface powered by <a href="">asyncio</a> for <a href="">peewee</a> ORM.</li> </ul></li> </ul> <div class="codehilite"><pre><span></span><code> <span class="n">database</span><span class="o">.</span><span class="n">set_allow_sync</span><span class="p">(</span><span class="bp">False</span><span class="p">)</span> <span class="n">async</span> <span class="k">def</span> <span class="nf">handler</span><span class="p">():</span> <span class="n">await</span> <span class="n">objects</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="n">TestModel</span><span class="p">,</span> <span class="n">text</span><span class="o">=</span><span class="s2">&quot;Not bad. Watch this, I&#39;m async!&quot;</span><span class="p">)</span> <span class="n">all_objects</span> <span class="o">=</span> <span class="n">await</span> <span class="n">objects</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">TestModel</span><span class="o">.</span><span class="n">select</span><span class="p">())</span> <span class="k">for</span> <span class="n">obj</span> <span class="ow">in</span> <span class="n">all_objects</span><span class="p">:</span> <span class="k">print</span><span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="n">text</span><span class="p">)</span> </code></pre></div> <p><strong>Brian #5:</strong> <a href=""><strong>Machine Learning Basics</strong></a></p> <ul> <li>“Plain python implementations of basic machine learning algorithms”</li> <li>From the repo: <ul> <li>A repository of implementations of basic machine learning algorithms in plain Python (Python Version 3.6+). All algorithms are implemented from scratch without using additional machine learning libraries. The intention of these notebooks is to provide a basic understanding of the algorithms and their underlying structure, not to provide the most efficient implementations. <ul> <li>Linear Regression</li> <li>Logistic Regression</li> <li>Perceptron</li> <li>k-nearest-neighbor</li> <li>k-Means clustering</li> <li>Simple neural network with one hidden layer</li> <li>Multinomial Logistic Regression</li> </ul></li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Cerberus</strong></a></p> <ul> <li>Cerberus provides powerful yet simple and lightweight data validation functionality out of the box</li> <li>designed to be easily extensible, allowing for custom validation</li> <li>Origin of the name: CERBERUS, n. The watch-dog of Hades, whose duty it was to guard the entrance;</li> </ul> <div class="codehilite"><pre><span></span><code> <span class="n">schema</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;type&#39;</span><span class="p">:</span> <span class="s1">&#39;string&#39;</span><span class="p">},</span> <span class="s1">&#39;age&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;type&#39;</span><span class="p">:</span> <span class="s1">&#39;integer&#39;</span><span class="p">,</span> <span class="s1">&#39;min&#39;</span><span class="p">:</span> <span class="mi">10</span><span class="p">}}</span> <span class="n">v</span> <span class="o">=</span> <span class="n">Validator</span><span class="p">(</span><span class="n">schema</span><span class="p">)</span> <pre><code>&lt;span class="n"&gt;document&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;name&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;Little Joe&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;age&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="n"&gt;v&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;validate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;document&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c1"&gt;# False&lt;/span&gt; </code></pre> <span class="n">v</span><span class="o">.</span><span class="n">errors</span> <span class="c1"># {&#39;age&#39;: [&#39;min value is 10&#39;]}</span> </code></pre></div> <p><strong>Follow up and other news</strong></p> <p><strong>Michael:</strong></p> <p><strong>#100DaysOfCode in Python course</strong>: <strong><a href=""></a></strong></p>
Mar 23, 2018
#69 Digging into StackOverflow's 2018 survey results
<h1>Python Bytes 69</h1> <p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>pynb: Jupyter Notebooks as plain Python code with embedded Markdown text</strong></a></p> <ul> <li><code>pynb</code> lets you manage Jupyter notebooks as plain Python code with embedded Markdown text, enabling: <ul> <li><strong>Python development environment</strong>: Use your preferred IDE/editor, ensure style compliance, navigate, refactor, and test your notebooks as regular Python code.</li> <li><strong>Version control</strong>: Track changes, review pull requests and merge conflicts as with regular Python code. The cell outputs are stored separately and don't interfere with versioning.</li> <li><strong>Consistent execution state</strong>: Never lose track again of the execution state. Notebooks are always executed from clean iPython kernels and the cell execution is cached.</li> </ul></li> <li>You also get parameterized notebooks with batch and programmatic execution.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Microsoft’s quantum computing language is now available for</strong></a> <a href=""><strong>m</strong></a><a href=""><strong>acOS</strong></a></p> <ul> <li>New language Q# (<a href="">snippet examples</a>)</li> <li>How do you run a quantum app?</li> <li>Based on topological qubits and quantum computers</li> <li>Now out on macOS &amp; Linux</li> <li>Need to use VS Code (and vs code extension)</li> <li>Comes with Python interoperability (only other language)</li> <li>Also in Jupyter</li> <li>Some real-world examples. See <a href="">this Wired article</a>. <ul> <li><a href="">D-wave</a></li> <li>IBM is making quantum computers commercially available. Since 2016, it has offered researchers the chance to run experiments on a five-qubit quantum computer via the cloud and at the end of 2017 started making its 20-qubit system available online too.</li> </ul></li> </ul> <p><strong>Brian #3: pytest talk in Spanish</strong> </p> <ul> <li>"pytest: recommendations, basic packages for testing in Python and Django"</li> <li>By A. Vallbona (<a href="">@avallbona</a>) From PyConES 2017</li> <li>with <a href="">English slides</a>, and <a href=";">video in Spanish</a>.</li> <li>Some of the topics covered: <ul> <li><code>pytest-django</code></li> <li><code>model-mommy</code> to easily create fixtures based on django models</li> <li><code>pytest-lazy-fixture</code> allows the use the fixtures as parameters to parameterize</li> <li><code>pytest-mock</code>, <code>pytest-cov</code>, <code>pytest-flake8</code></li> <li><code>freezegun</code> to helps us to "freeze" time</li> <li><code>eradicate</code> to eliminate commented code</li> <li><code>pytest-xdist</code> to run tests in parallel</li> </ul></li> </ul> <p><strong>Bonus pytest topic:</strong></p> <ul> <li><a href=""> just added a Reference page</a>, a full reference to pytest’s API.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>StackOverflow Developer Survey Results 2018</strong></a></p> <ul> <li>Sample size: Over 100,000 developers</li> <li>55% contribute to open source</li> <li>64% have CS degrees</li> <li>Experience and Belonging <ul> <li>Connection to other devs (increasing over time)</li> <li>Competing with peers (decreasing over time)</li> <li>Not as good as my peers (decreasing over time)</li> </ul></li> <li>How Much Time Do Developers Spend on a Computer? Most: 9-12 hours</li> <li>Python beats C# in usage for the first time</li> <li>Languages: <ul> <li>Most loved: #1 Rust, #2 Kotlin, #3 Python</li> <li>Most dreaded: VB 6 and CoffeeScript</li> <li>Most wanted: #1 Python 25%, #JavaScript 19%, #3 Go 16%</li> </ul></li> <li>Databases: <ul> <li>Loved: PostgreSQL</li> <li>Dreaded: IBM Db2, Memcached, and Oracle</li> <li>Most wanted: MongoDB</li> </ul></li> <li>Editor: VS Code</li> <li>Dev OSes: <ul> <li>Windows: 49%</li> <li>macOS: 27%</li> <li>Linux: 23%</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>demoshell</strong></a></p> <ul> <li><a href=""><strong>@</strong></a><a href="">doughellmann</a></li> <li>Doug Hellman (<a href=""><strong>@</strong></a><a href="">doughellmann</a>) <a href="">announces demoshell</a> <ul> <li>Inspired by a tweet from <a href="">@genehack</a> “Hey, speakers, if you're doing live demos in a shell, clear the screen after <em>every</em> <em>command</em> to get the prompt back at the top, so folks in the back can see what you're doing.”</li> </ul></li> <li>demoshell is a simplified shell for live demonstrations. It always shows the command prompt at the top of the screen and pushes command output down instead of letting it scroll up.</li> <li>In his words: “I put it up there to start a discussion. I’d be happy if a bunch of people showed up and wanted to take it over and actually turn it into something useful. I invite people to give it a look. And warn them that too much interest is going to be met with commit privileges on the repo. :-)”</li> </ul> <p><strong>Michael #6: Clear statement on Python 2 EOL</strong></p> <ul> <li>Will there be a period where Py2.7 is in security-only status before hitting EOL?</li> <li>via Nicola Iarocci‏ <a href="">@nicolaiarocci</a> <ul> <li>Yay, @gvanrossum makes it adamantly clear: “Let's not play games with semantics. The way I see the situation for <strong>2.7 is that EOL is January 1st, 2020</strong>, and there will be no updates, <strong>not even source-only security patches</strong>, after that date.” <strong></strong> </li> </ul></li> <li>Support (from the core devs, the PSF, and stops completely on that date.</li> </ul> <p><strong>Follow up and other news</strong></p> <p><strong>Michael:</strong></p> <ul> <li>Eve: MongoDB &amp; Flask-backed RESTful APIs course is out! <ul> <li><strong><a href=""></a></strong></li> </ul></li> <li>Shoutout to everyone I met at <a href="">PyCon Slovakia</a></li> </ul> <p><strong>Brian:</strong> </p> <ul> <li>A couple of recent episodes on Test &amp; Code focusing on project test development: <ul> <li><a href="">What tests to write first</a> </li> <li><a href="">Prioritize software tests with RCRCRC</a></li> <li>Upcoming topics will include beefing up test coverage with things like equivalence partitioning, boundary value analysis, state transition diagrams, state tables, negative testing, …</li> <li>Also learning a lot about developing an open source project and all the tools surrounding that. I’ll discuss those topics in episodes as well.</li> <li>Project used in both episodes, <a href="">cards : a project task tracking / todo list app</a> that will be expanded as I go along talking about different test and software development topics.</li> </ul></li> </ul>
Mar 18, 2018
#68 Python notebooks galore!
<p>Sponsored by DigitalOcean! <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>dumb-pypi</strong></a></p> <ul> <li>This takes some fiddling with and trial and error. I definitely need to write up my experiences with this as a blog post.</li> <li>Combine with <code>pip download</code> (covered in <a href="">episode 24</a>), this makes it super easy to create a static locally hosted pypi server, either for all of your packages, or for your proprietary packages.</li> <li>Roughly:</li> </ul> <pre><code> pip download -d my-packages-dir [HTML_REMOVED] ls my-packages-dir &gt; package-list.txt dumb-pypi --package-list my-packages-dir \ --packages-url [HTML_REMOVED] \ --output-dir my-pypi </code></pre> <ul> <li>Now add something like this to requirements.txt or pip commands:</li> <li><code>--trusted-host [HTML_REMOVED] -i http://[HTML_REMOVED]/my-pypi/simple</code></li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Requests-HTML: HTML Parsing for Humans</strong></a></p> <ul> <li>This library intends to make parsing HTML (e.g. scraping the web) as simple and intuitive as possible.</li> <li>When using this library you automatically get: <ul> <li>Full JavaScript support!</li> <li>CSS Selectors (a.k.a jQuery-style, thanks to PyQuery).</li> <li>XPath Selectors, for the faint at heart.</li> <li>Mocked user-agent (like a real web browser).</li> <li>Automatic following of redirects.</li> <li>Connection–pooling and cookie persistence.</li> <li>The Requests experience you know and love, with magical parsing abilities</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>A phone number proxy</strong></a></p> <ul> <li>Naomi Pentrel, <a href="">@naomi_pen</a> on twilio blog</li> <li>Set up a phone number that you can share for temporary events to send and receive texts that get forwarded to your actual number.</li> </ul> <p><strong>Michael #4: Notebooks galore part 1:</strong> <a href=""><strong>Datalore</strong></a></p> <ul> <li>In cloud and ready to go</li> <li>Intelligent code editor</li> <li>Out-of-the-box Python tools</li> <li>Collaboration</li> <li>Integrated version control</li> <li>Incremental calculations: Improve and adjust models without hustling with additional recalculations. Datalore follows dependencies between multiple computations and automatically applies relevant recalculations.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>bellybutton</strong></a></p> <ul> <li>by Chase Stevens, <a href="">@hchasestevens</a></li> <li>Tool for creating personal static analysis/style tools like <code>pycodestyle</code>, <code>pylint</code>, and <code>flake8</code></li> <li>Teams often have some of their own style requirements that can’t be expressed as <code>flake8</code> flags and exceptions.</li> <li>Example: deprecating internal library functions and catching that by the linter. </li> </ul> <p><strong>Michael #6:Notebooks galore part 2</strong></p> <ul> <li><a href=";sh=0c41688d">Python 3.6 Jupyter Notebook on Azure</a></li> <li><a href="">Google Colaboratory</a></li> <li><a href="">JupyterLab is Ready for Users</a> <ul> <li>JupyterLab is an interactive development environment for working with notebooks, code and data. Most importantly, JupyterLab has full support for Jupyter notebooks. Additionally, JupyterLab enables you to use text editors, terminals, data file viewers, and other custom components side by side with notebooks in a tabbed work area.</li> </ul></li> <li>you can pip install python packages within python code itself. <ul> <li>Super useful in situation #1 when you need a package that's not included but you don't have access to the shell. </li> <li>If you need to upgrade a package. For example the Pandas version is a little old on Azure, so you can upgrade by simply running:</li> </ul></li> </ul> <pre><code> import pip pip.main(['install', 'pandas', '--upgrade']) </code></pre>
Mar 06, 2018
#67 Result of moving Python to Github
<p>Sponsored by DigitalOcean! <a href="">http://<strong></strong></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>Building a blog with Pelican</strong></a></p> <ul> <li>We did cover Pelican in <a href="">episode 38</a>, but this is a nice tutorial in 7 parts on building a blog.</li> <li>Peter Kazarinoff, <a href="">@pkazarinoff</a> <ul> <li>Nice blog with a focus on engineering students.</li> </ul></li> <li>Starts with installing Python and git and some other tools.</li> <li>Step by step, every action to get a a blog up as a static site hosted on github pages.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Notifiers</strong></a> </p> <ul> <li>Got an app or service and you want to enable your users to use notifications with their provider of choice? </li> <li>Working on a script and you want to receive notification based on its output? </li> <li>A one stop shop for all notification providers with a unified and simple interface.</li> <li>A unified interface means that you already support any new providers that will be added, no more work needed!</li> <li>Some providers <ul> <li>Slack</li> <li>Gmail</li> <li>Telegram</li> <li>Gitter</li> <li>…</li> </ul></li> <li>Python 3 only</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Using Makefiles in Python projects</strong></a></p> <ul> <li>Krzysztof Żuraw, <a href="">@krzysztof_zuraw</a></li> <li>Alerted to this article from kidpixo, <a href="">@kidpixo</a></li> <li>We don’t usually think of Makefiles and Python together, but they can be a handy place to keep common scripts for a project all in one place.</li> <li>This article is a nice gentle intro to Makefiles and shows some cool uses: <ul> <li>cleaning out .pyc files</li> <li>cleaning out egg directories</li> <li>linting</li> <li>running tests with flags</li> <li>starting a test server</li> <li>deploying</li> <li>sorting import files</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Result of moving Python to Github</strong></a></p> <ul> <li>See the graph linked in the post</li> <li>A couple of quick numbers (including PRs too) from 2017 compared to 2016: <ul> <li>the number of commit has increased by 190%</li> <li>inserted lines of code has increased by 140%</li> <li>number of unique contributors has increased by 1300%</li> <li>number of returning contributors has increased by 900%</li> </ul></li> <li>One comment was: “Personally, I would like them moving to Gitlab instead, but still good news.” I tend to disagree.</li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Self-Deprecation Needs to</strong></a> <a href=""><strong>Stop</strong></a></p> <ul> <li>Maurice Hayward, <a href=""><strong>@</strong></a><a href="">maurice_hayward</a></li> <li>Inspired by some tweets by Stephanie Hurlburt, <a href=""><strong>@</strong></a><a href="">sehurlburt</a></li> <li><strong>Stop saying these words when describing yourself or your accomplishments.</strong> These words are now under BAN: "My project is..." - very small/basic/simple - not that good - a thing I wrote - just by a newbie - something I didn't spend a lot of time/effort on - silly - not that useful Just state the topic and let others be the judge.</li> <li><strong>Really think about the value you bring, then let everybody know.</strong></li> <li><strong>Be Proud of Yourself!</strong></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>5 speed improvements in Python 3.7</strong></a></p> <ol> <li><strong>Calling methods faster (maybe)</strong> <ol> <li>Python 3.7 adds 2 new Opcodes, LOAD_METHOD and CALL_METHOD for when the compiler sees x.method(...) it uses these new Opcodes.</li> <li>Bound methods with no arguments are now faster</li> </ol></li> <li><strong>str.find() is faster for some characters</strong> <ol> <li>Some unicode characters have an unfortunate issue when scanning a string for occurrences using str.find(x), seeing up to 25x slow down.</li> <li>These are still slower, but now 3x slower than ASCII characters instead of 25x!</li> </ol></li> <li><strong>os.fwalk is 2x faster</strong></li> <li><strong>Regular expressions are faster</strong> <ol> <li>A change was made in Python 3.6 which slowed down this call when flags were passed which were integers. Python 3.7 “fixes” the slowdown but is still not as fast as Python 3.5</li> </ol></li> <li><strong>Regular expressions are faster for case-insensitive matching</strong> <ol> <li>The speed improvement is significant, if you’re matching ASCII characters you can see up to a 20x improvements in matching time since it’s now doing a lookup instead of running lower() over each character.</li> </ol></li> </ol> <p>Follow up and other news</p> <p>Brian: </p> <ul> <li><a href="">Python package maintainers, help test the new PyPI!</a></li> <li><a href="">pytest/pycharm webinar is up</a>.</li> </ul>
Mar 01, 2018
#66 Wait, NoSQL with ACID and transactions?
<p>Sponsored by Rollbar: <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>Object-Oriented Programming (OOP) in Python 3</strong></a></p> <ul> <li>Real Python</li> <li>Nice modern introduction to classes, inheritance, and OOP.</li> <li>Classes, objects, attributes, instances, and inheritance.</li> <li>One gotcha not mentioned <ul> <li>The <code>__init__()</code> method of a base class is not called automatically by derived classes. If you override it, you need to call <code>super().__init__()</code>.</li> </ul></li> <li>Also, check out <a href="">attrs</a> for much of our OOP needs</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>ScriptedForms</strong></a></p> <ul> <li>Quickly create live-update GUIs for Python packages using Markdown and a few custom HTML elements. </li> <li>Just write in markdown + variables / UI types</li> <li>Based on Jupyter</li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>MongoDB to add multi-document transactions and ACID</strong></a></p> <ul> <li>Mind blown. Didn’t see this coming</li> <li>“MongoDB 4.0 will add support for multi-document transactions, making it the only database to combine the speed, flexibility, and power of the document model with ACID data integrity guarantees. Through snapshot isolation, transactions provide a globally consistent view of data, and enforce all-or-nothing execution to maintain data integrity.”</li> <li>Due out this summer.</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Python packaging pitfalls</strong></a></p> <ul> <li>Just a short list of packaging blunders <ul> <li>Forgetting to clean the build dir</li> <li>Forgetting to specify package data</li> <li>Fine grained</li> <li>Using package_data, or worse: fine grained package_data</li> <li>Listing excludes/prunes before includes/grafts</li> <li>Hardcoding packages list in</li> <li>Hardcoding py_modules list in</li> <li>Importing your package in</li> <li>Importing unavailable tools in</li> <li>Messing with the environment</li> <li>Your tests do not test the installed code</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Blogging principles</strong></a></p> <ul> <li>Julia Evans @b0rk</li> <li>Be honest about what you know</li> <li>Try not to write anything too long. <ul> <li>(My note: don’t shy away from long tutorials. Just don’t only do long stuff)</li> </ul></li> <li>Be positive.</li> <li>Write for the past you. </li> <li>Stick with your own experience.</li> <li>It’s ok if not everyone likes it <ul> <li>Don’t try to keep one upping yourself.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>pipenv is officially official</strong></a></p> <ul> <li>Pipenv — the officially recommended Python packaging tool from, free (as in freedom). <ul> <li>Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. </li> <li>Windows is a first–class citizen, in our world.</li> </ul></li> <li>Benefits? <ul> <li>It automatically creates and manages a virtualenv for your projects</li> <li>adds/removes packages from your Pipfile as you install/uninstall packages</li> <li>generates the ever–important Pipfile.lock, which is used to produce deterministic builds.</li> </ul></li> </ul> <p>Follow up and other news</p> <p><strong>Brian</strong></p> <ul> <li>Productive pytest with PyCharm webinar was recorded Thursday 22nd of Feb. </li> <li>Will be available here: <a href=""><strong></strong></a></li> </ul> <p>Michael:</p> <ul> <li>Embed Python in Unreal Engine 4 <strong><a href=""></a></strong></li> <li>Pandas documentation sprint <strong><a href=""></a></strong></li> </ul>
Feb 23, 2018
#65 Speed of your import statements affecting performance?
<p>Sponsored by Rollbar: <a href=""></a></p> <p><strong>Brian #1:</strong> <a href=""><strong>pygal</strong></a> <strong>: Simple Python Charting</strong> </p> <ul> <li>Output SVG or PNG</li> <li>Example Flask App (also django response) part of documentation.</li> <li>Enough other bits of doc to get you a chart in a web page super fast.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>Thoughts on becoming a self-taught programming</strong></a></p> <ul> <li>Basic format:</li> <li>I'm 31 days into self-studying Python and am loving every minute of it!</li> <li>A few questions: <ul> <li>What were you doing before you began self-studying programming?</li> <li>What made you want to study programming on your own?</li> <li>How did you start (which resources and language)?</li> <li>How long did it take for you to feel confident enough in your skills and knowledge to know you could be employed as a programmer?</li> <li>What else did you do besides self-study that helped you in your journey to becoming a programmer?</li> <li>What's next for you?</li> </ul></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>How to speed up Python application startup time (timing imports in 3.7)</strong></a></p> <ul> <li>Python 3.7 includes <code>-X importtime</code> option that allows you to profile the time it takes to do all the imports.</li> <li>Way cool tool to help optimize the startup time of an application.</li> </ul> <p><strong>Michael #4:</strong> <a href="">AnPyLar - The Python web front-end framework</a></p> <ul> <li>Create web applications with elegance, simplicity and yet full power with Python and components</li> <li>MISSION: Empower all Python programmers to work not only on the back-end but also on the front-end with the same language of choice</li> <li>Features <ul> <li>Reactive programming and Promises</li> <li>Python standard formatting as templates</li> <li>reusable components</li> <li>Scoped styling for component</li> <li>Integrated routing engine</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Migrating to Python 3 with pleasure</strong></a></p> <ul> <li><strong>“A short guide on features of Python 3 for data scientists”</strong></li> <li>Quick tutorial through examples of <code>pathlib</code>.</li> <li>Type hinting and how cool it works with editors (PyCharm example shown)</li> <li>Adding runtime type enforcement for specific methods using <a href="">enforce</a></li> <li>Using function annotations for units, as done in <a href="">astropy</a>. </li> <li>Matrix multiplication with <code>@</code>.</li> <li>Globbing with <code>**</code>. <ul> <li><code>found_images = glob.glob('/path/**/*.jpg', recursive=True)</code></li> </ul></li> <li>Also … underscores in numeric literals, f-strings, true division with <code>/</code>, integer division with <code>//</code>, and lots of more fun goodies.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Moving to Python 3</strong></a></p> <ul> <li>Many of these issues were corrected just by running 2to3, which not only fixed many of the compatibility issues <ul> <li>Outdated external libraries which needed to be updated to newer versions featuring Python 3 compatibility <code>basestring</code> to <code>str</code>, <code>urlparse</code> to <code>urllib.urlparse</code> and similar major changes</li> <li>Dictionary change like <code>iteritems()</code> to <code>items()</code>, or <code>.items()</code> now returning a view.</li> <li>Things that weren't needed anymore, like Django's <code>force_unicode</code> or <code>__future__</code> library tools.</li> </ul></li> <li>Once we finished working on the "low-hanging fruits", the next step was to run Aphrodite's test suite and achieve zero errors.</li> <li>Lessons learned <ul> <li>Code coverage was originally around 70%,</li> <li>Keeping the Python 3 branch up to date with master</li> <li>A non-trivial feature was delivered during the migration (via feature branch)</li> <li>The pickle protocol version in python 3 can be higher than the highest available in Python 2.7. So we needed to add versioning to our Django caches</li> <li>Each modified file had to comply with flake8 linting rules</li> </ul></li> <li>Afrodita is currently running on Google App Engine Flexible, and one of the features our team loves with is traffic splitting</li> <li>With this feature, we can do <a href="">canary releases</a> with ease: We just deploy our new version of the service, and start redirecting small amounts of traffic traffic while we monitor for unexpected errors.</li> <li>After some minor bugfixes, we could bring the traffic of the Python 3.6 version to 100% with confidence. We also had the old version available for instant rollback, thanks to how parallel versions and traffic splitting work in GAE flexible.</li> </ul> <p><strong>Our news</strong></p> <p>Brian:</p> <ul> <li>Upcoming webinar: <a href="">Productive pytest with Pycharm</a></li> </ul> <p>Michael:</p> <ul> <li>My GUI example: <a href=""></a></li> </ul>
Feb 14, 2018
#64 The GUI phoenix rises with wxPython
<p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p><strong>Brian #1: wxPython 4,</strong> <strong>Pheonix is now live and supports Python 3</strong></p> <ul> <li><a href="">wxPython on PyPI</a></li> <li><a href="">4.0.0</a>, <a href="">4.0.1 release notes</a></li> <li>If you haven’t played with wxPython for a while, now might be a good time.</li> </ul> <p><strong>Michael #2:</strong> <a href=""><strong>typeshed</strong></a></p> <ul> <li>Typeshed contains external type annotations for the Python standard library and Python builtins, as well as third party packages.</li> <li>This data can e.g. be used for static analysis, type checking or type inference. </li> <li>Used as the basis of mypy and PyCharm’s magic</li> <li>Each Python module is represented by a .pyi "stub". This is a normal Python file (i.e., it can be interpreted by Python 3), except all the methods are empty. Python function annotations (PEP 3107) are used to describe the types the function has.</li> <li>Here’s what one of these exeternal definitions looks like:</li> </ul> <pre><code> class NodeVisitor(): def visit(self, node: AST) -&gt; Any: ... def generic_visit(self, node: AST) -&gt; None: ... </code></pre> <p><strong>Brian #3:</strong> <a href=""><strong>Coverage 4.5 adds configurator plug-ins</strong></a></p> <ul> <li>“There’s one new feature: <a href="">configurator plug-ins</a>, that let you run Python code at startup to set the configuration for coverage. This side-steps a requested feature to have different exclusion pragmas for different versions of Python.”</li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>Python integrated into Unreal Engine</strong></a></p> <ul> <li>via Pirogov Alexander‏ ( <a href="">@Pie_Daddy</a> )</li> <li>tl;dr: Autodesk university plans to integrate Python into Unreal Engine for the data integration pipeline and ease the process of bringing assets into the game.</li> <li>Autodesk is working on that will solve complicated problems with bringing CAD data into the Unreal Engine.</li> <li>Where they are today: <ul> <li>The Datasmith workflow toolkit, currently in beta, makes moving data into Unreal Engine as frictionless as possible. </li> <li>Datasmith provides high-fidelity translation of common scene assets such as geometry, textures, materials, lights and cameras from popular DCC and CAD applications into Unreal Engine.</li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>Python 3.7.0b1</strong></a> <strong>: Beta means we should be testing it!!!</strong></p> <ul> <li>If not people like us and our listeners, then who? Seems like we’re a good set of beta testers.</li> <li>What are you going to test?</li> <li>I'm going to look at breakpoint() and data classes.</li> </ul> <p><strong>Michael #6: Releases abound!</strong></p> <ul> <li>Django security releases issued: 2.0.2 and 1.11.10 <a href=""></a></li> <li>Python 3.4.8 (security) <a href=""></a></li> <li>Python 3.5.5 (security) <a href=""></a> <ul> <li>libexpat XML lib had a security issue</li> </ul></li> <li>Channels 2.0 is a major rewrite of Channels <ul> <li>See <a href="">Talk Python’s interview</a> for more details</li> <li>Notably: Python 2.7 and 3.4 are no longer supported.</li> </ul></li> </ul> <p><strong>Our news</strong></p> <p>Brian:</p> <ul> <li>Speaking at PyCon 2018. “PyCharm and pytest”. Speaking with Paul Everitt</li> <li>Upcoming webinar: <a href="">Productive pytest with Pycharm</a> <ul> <li>Feb 22, registration open</li> </ul></li> </ul> <p>Michael:</p> <ul> <li><a href=""><strong>Webinar Recording:</strong></a> “MongoDB Quickstart with Python and PyCharm” with Michael Kennedy</li> </ul>
Feb 09, 2018
#63 We're still on a desktop GUI kick
<p>Sponsored by DigitalOcean: <strong><a href=""></a></strong></p> <p><strong>Brian #1:</strong> <a href=""><strong>A brief tour of Python 3.7 data classes</strong></a></p> <ul> <li>a great write-up of the upcoming data classes via Anthony Shaw</li> <li>“Data classes are a way of automating the generation of boiler-plate code for classes which store multiple properties. They also carry the benefit of using Python 3’s new type hinting.”</li> <li>Default magic methods <ul> <li>In the default setting, any dataclass will implement <code>__init__</code>, <code>__repr__</code>, <code>__str__</code> and <code>__eq__</code> for you.</li> <li>The <code>__init__</code> method will have <strong>keyword-arguments</strong> with the same type annotations that are specified on the class.</li> <li>The <code>__eq__</code> method will compare all dataclass attributes in order.</li> <li>All fields are declared at the top of the class and type hinting is <strong>required</strong>.</li> </ul></li> <li>Also covered <ul> <li>type hinting</li> <li>mutability (and frozen)</li> <li>customizing the fields</li> <li>post-init processing : optional <code>__``*post_init_*``_</code> will run after the generated <code>_``*_init_*``_</code></li> <li>inheritance</li> </ul></li> </ul> <p><strong>Michael #2:</strong> SQLite [The Databaseology Lectures - CMU Fall 2015]</p> <ul> <li>Lots of DBs covered here: <strong><a href=""></a></strong></li> <li>SQLite at this <a href=";index=2&amp;list=PLSE8ODhjZXjakeQR57ZdN5slUu2oPUr1Y"><strong>YouTube video</strong></a></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>dryable</strong></a> <a href=""><strong>:</strong></a> <a href=""><strong>a useful dry-run decorator for python</strong></a></p> <ul> <li>short circuit methods within your project during dry runs.</li> <li>example shows how to add a command line flag <code>--dry-run</code>.</li> <li>The test code is useful for understanding it also.</li> <li><p>Example import dryable</p> <p>@dryable.Dryable('foo') def return_something(): return 'something'</p></li> </ul> <p></p> <pre><code>from something import return_something import dryable def test_normal_return(): dryable.set(False) assert return_something() == 'something' def test_dry_return(capsys): dryable.set(True) assert return_something() == 'foo' </code></pre> <p><strong>Michael #4:</strong> </p> <ul> <li>These are some pretty cool examples. <ul> <li><a href=""></a></li> <li><a href=""></a></li> <li><a href=""></a></li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href=""><strong>PEP Explorer - Explore Python Enhancement Proposals</strong></a></p> <ul> <li>Cool idea. Might need some work though. I can’t find any accepted PEPs for 3.7, including <a href="">557, data classes</a>.</li> <li>I’m ok with giving Anthony some shade on this, as we highlighted his writing in the first item.</li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>TKInter Tutorial</strong></a></p> <ul> <li>via @likegeeks</li> <li>Create your first GUI application</li> <li>Create a label and button widgets</li> <li>Input and combo boxs, menus, progressbars and more</li> </ul> <h2>Our news</h2> <p>Michael</p> <ul> <li>I built something with <a href="">Gooey</a> this weekend, it was wonderful.</li> <li>Self-serve team purchases and discounts at Talk Python Training</li> </ul>
Feb 01, 2018
#62 Wooey and Gooey are simple Python GUIs
<p>Brought to you by Datadog <a href=""></a></p> <p><strong>Brian #1:</strong> <strong>Dan Bader takes over Real Python</strong></p> <ul> <li><a href="">Announcement email</a>, with what Michael, Fletcher, and Jeremy are doing now</li> <li>Dan is on the show and tells us all about it.</li> </ul> <p><strong>Michael #2: Still more Python GUIs</strong></p> <ul> <li><strong><a href=""></a></strong> <ul> <li>A Django app that creates automatic web UIs for Python scripts.</li> <li>Wooey is a simple web interface to run command line Python scripts. Think of it as an easy way to get your scripts up on the web for routine data analysis, file processing, or anything else.</li> <li>Wooey was envisioned as a system to allow data analysts to be able to easily: <ul> <li>Autodocument workflows for data analysis (simple model saving).</li> <li>Enable fellow co-workers with no command line experience to utilize python scripts.</li> <li>Enable the easy wrapping of any program in simple python instead of having to use language specific to existing tools such as Galaxy.</li> </ul></li> <li>Try the demo server: <a href=""></a></li> </ul></li> <li><strong><a href=""></a></strong> <ul> <li>Turn (almost) any Python command line program into a full GUI application with one line</li> <li>See the <a href=""><strong>screenshots here</strong></a></li> <li>Gooey converts your Console Applications into end-user-friendly GUI applications. </li> <li>It lets you focus on building robust, configurable programs in a familiar way, all without having to worry about how it will be presented to and interacted with by your average user.</li> </ul></li> <li>And Toga: <a href=""></a></li> </ul> <p><strong>Brian #3:</strong> <a href=""><strong>Python’s misleading readability</strong></a></p> <ul> <li>Ned Batchelder</li> <li><code>is</code> and <code>or</code> are not obvious and can confuse people new to the language, new to programming. <ul> <li><code>1000 + 1 is 1001</code> → <code>1000 + 1 == 1001</code></li> <li><code>answer == "y" or "yes``"</code> → <code>answer in {"y", "yes"}</code></li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href=""><strong>warp2 access</strong></a> </p> <ul> <li>python2 code from python3</li> <li>It communicates with the subprocess using pickle, so there are limitation to using it - if you need to send unpicklable data, that’s a problem.</li> </ul> <p><strong>Brian #5:</strong> <strong>Help! My tests can’t see my code!</strong></p> <ul> <li>Probably should be an episode on <a href="">Test &amp; Code</a>, and maybe I’ll do that also, but it’s a big enough roadblock to many newcomers to <a href="">pytest</a>, that I want to get the word out on how to fix it.</li> <li>A best practice is to put your test code in a folder called tests.</li> <li>Now, if you are sitting in the parent directory, where you can see both your modules/packages under test and the <code>tests</code> directory, and you run <code>pytest</code>, your test code has to have some way to import the code under test.</li> <li>If you are in a hurry. Homework due in an hour, project manager breathing down your neck, or whatever, then there are two easy options: <ul> <li><code>python -m pytest</code></li> <li><code>python</code> adds the current directory where you start it to <code>PYTHONPATH</code>, <code>pytest</code> does not.</li> <li><code>pip install pytest-pythonpath</code></li> <li><a href=""></a></li> <li>This plugin adds the current directory to <code>PYTHONPATH</code>, and adds some hooks that let you define search paths in your <code>pytest.ini</code> file.</li> </ul></li> <li>When you have time.. <ul> <li>Create a <code></code> file for your code. And…</li> <li><code>pip install -e ./your_project</code></li> <li>This allows you to continue working on your code while letting your test code see the code under test</li> <li>This method is friendlier to <code>tox</code>.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href=""><strong>Cement - Framework for CLI</strong></a></p> <ul> <li>Cement is an advanced CLI Application Framework for Python. </li> <li>Its goal is to introduce a standard, and feature-full platform for both simple and complex command line applications </li> <li>Also supports rapid development needs without sacrificing quality.</li> <li>Core features <ul> <li>Core pieces of the framework are customizable via handlers/interfaces</li> <li>Extension handler interface to easily extend framework functionality</li> <li>Config handler supports parsing multiple config files into one config</li> <li>Argument handler parses command line arguments and merges with config</li> <li>Log handler supports console and file logging</li> <li>Plugin handler provides an interface to easily extend your application</li> <li>Hook support adds a bit of magic to apps and also ties into framework</li> <li>Handler system connects implementation classes with Interfaces</