November 29, 2019

Weekend long read suggestions 2019/48

Machine Learning

  • Illustrated: Self-Attention

    Transformer-based architectures, which are primarily used in modelling language understanding tasks, eschew the use of recurrence in neural network and instead trust entirely on self-attention mechanisms to draw global dependencies between inputs and outputs. This article provides visual guide for understanding of this key paradigm.

  • How to recognize AI snake oil

    Handouts from Arvind Narayanan’s talk showing a great deal of examples where is ML/AI used only to increase perceived value of the project/service or is just pure bs. He also points to outcomes of such behavior – Hunger for personal data; Lack of explainability leading to wrong or black-box decisions and many others.

  • Machine Learning on Encrypted Data Without Decrypting It

    Very interesting article showing ways how you can train model without decrypting the actual data.

  • Lessons learned from building practical deep learning systems

    Slides from a guest lecture at the Fullstack Deep Learning Bootcamp by Xavier Amatriain. Topics covered are:

    1. More Data or Bette Data?
    2. Simple Models >> Complex Models
    3. But, sometimes, you need a Complex Model
    4. You should care about Feature Engineering
    5. Supervised and Unsupervised learning
    6. Everything is an ensemble
    7. There are biases in your data
    8. Think about your models “in the wild”
    9. The right evaluation approach.
    10. Value of systems and frameworks
    11. Machine Learning Infrastructure
    12. ML beyond Deep Learning
  • The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design

    Essay from Jeffrey Dean of Google Research about specialized hardware for machine learning, TPUs in particular.

  • DATA 1010 Cheat sheet

    Beautifully done cheat sheet from Brown University course on data science.


  • Abusing HTTP hop-by-hop request headers

    Introduction into no-so-well-known set of HTTP headers. HTTP headers at typically end-to-end, designed to be present in the request all the way. Hop-by-hop headers are designed to be processed and consumed by the proxy currently handling the request.

  • You cannot cURL under pressure

    Shedding some light into infrequently used curl features. Game gives you challenges you need to solve with curl. You will not only fetch webpage from known URL but also send emails and much more. Ben also open-sourced code behind the whole game.


  • Reverse Engineering for Beginners

    Absolutely massive open (CC BY-SA 4.0) book (1079 pages) introducing into reverse engineering of software. Starting with assembler basics, showing how the key constructs are compiled. Continuing to decompiling Java code. And wrapping with real-word examples code and file format disassembling.

  • A Python Interpreter written in Rust

    Just wow. When I’ve shared this within our team, one of our team members reacted with (thank you Tomas):

    Nice:) It reminded me this video of David Beazley “In this talk, I live-code a simple stack machine and turn it into an interpreter capable of running WebAssembly. I then use that to play a game written in Rust.” You can watch it on YoutTube

  • RubyConf 2019 - Adopting Sorbet at Scale by Ufuk Kayserilioglu

    Ufuk shares Shopify’s experience with adopting Sorbet (type checker for Ruby) into their massive code-base.