My Solution

In the first chapter itself, there are chips that you have to design for which you have to write a lot of repetitive code.

I am talking about the Or16, And16, and Not16 chips.

I will give one example. The Or16.hdl file that implements the 16-way Or gate looks like this:

// This file is part of www.nand2tetris.org
// and the book "The Elements of Computing Systems"
// by Nisan and Schocken, MIT Press.
// File name: projects/01/Or16.hdl

/**
 * 16-bit bitwise Or:
 * for i = 0..15 out[i] = (a[i] or b[i])
 */

CHIP Or16 {
    IN a[16], b[16];
    OUT out[16];

    PARTS:
    Or (a=a[0], b=b[0], out=out[0]);
    Or (a=a[1], b=b[1], out=out[1]);
    Or (a=a[2], b=b[2], out=out[2]);
    Or (a=a[3], b=b[3], out=out[3]);
    Or (a=a[4], b=b[4], out=out[4]);
    Or (a=a[5], b=b[5], out=out[5]);
    Or (a=a[6], b=b[6], out=out[6]);
    Or (a=a[7], b=b[7], out=out[7]);
    Or (a=a[8], b=b[8], out=out[8]);
    Or (a=a[9], b=b[9], out=out[9]);
    Or (a=a[10], b=b[10], out=out[10]);
    Or (a=a[11], b=b[11], out=out[11]);
    Or (a=a[12], b=b[12], out=out[12]);
    Or (a=a[13], b=b[13], out=out[13]);
    Or (a=a[14], b=b[14], out=out[14]);
    Or (a=a[15], b=b[15], out=out[15]);
}

There is a lot of repetition!

However, once we understand the format, we can simply write two lines of Python, or a language of your choice to generate the repetitive code.

The Code

The code is trivial, and looks like this:

for i in range(0, 16):
    print(f'    Or (a=a[{i}], b=b[{i}], out=out[{i}]);')

This gives you the output that you need.

All you need to do now is just copy the code and paste it into your chip HDL file.

You can repeat the same process for the And, and Not gates as well.

Conclusion

This was such a trivial program and easy to write, and it saved me probably some minutes. But as I was doing this, I realized, Computer Science is a field of study that rewards your laziness- if you are being lazy in the correct way. The old ways of: “do not take shortcuts” do not apply here. Take shortcuts that save you time, money, or teach you new things.

Introduction

In this Notebook, trajectories of the Logistic Map [1] is plotted with Python.

import numpy as np
import matplotlib.pyplot as plt

def logistic_function(parameter: float, initial_value: float) -> float:
    return parameter * initial_value * (1 - initial_value)

def logistic_function_trajectory(parameter: float,
                                 initial_value: float,
                                 num_iter: int) -> np.array:
    trajectory = np.zeros(num_iter)
    trajectory[0] = initial_value
    for i in range(num_iter-1):
        trajectory[i+1] = logistic_function(parameter, initial_value)
        initial_value = trajectory[i+1]
    return np.array(trajectory)

logistic_function_trajectory(parameter=2.2, initial_value=0.8, num_iter=10)

array([0.8       , 0.352     , 0.5018112 , 0.54999278, 0.54450159,
       0.54564314, 0.54541675, 0.5454621 , 0.54545303, 0.54545485])

xs = np.linspace(1, 200, 200)
ys = logistic_function_trajectory(parameter=2.2, initial_value=0.8, num_iter=200)

def plot_trajectory(xs, ys, x_label, y_label, title):
    plt.figure(figsize=(20,10))
    plt.scatter(xs, ys, marker='o', edgecolor='black', c=point_colors * 50);
    plt.xlabel(x_label);
    plt.ylabel(y_label);
    plt.title(title);
    plt.grid();

point_colors = ['cyan', 'magenta', 'yellow', 'black']

plot_trajectory(xs=xs, ys=ys, x_label='$n$', y_label='$x_n$', title='Trajectory')

Plotting vs.  for

x_n = logistic_function_trajectory(parameter=3.4, initial_value=0.2, num_iter=200)
x_n_hat = logistic_function_trajectory(parameter=3.4, initial_value=0.200001, num_iter=200)

ys = abs(np.subtract(x_n, x_n_hat))
xs = np.linspace(1, 200, 200)

plot_trajectory(xs=xs, ys=ys, x_label='$n$', y_label='$ x_n - \hat{x_n} $', title='Trajectory for $r = 3.4$')

We can thus plot the trajectories of the Logistic Map with Python and relevant libraries.

References

1. https://en.wikipedia.org/wiki/Logistic_map
2. Bradley, Liz: Nonlinear Dynamics: Mathematical and Computational Approaches via Complexity Explorer, Santa Fe Institute

Cite this Notebook:

@ONLINE {,
    author = "Ritobrata Ghosh",
    title  = "Non Linear Dynamics Trajectories with Python",
    month  = "apr",
    year   = "2023",
    url    = "https://www.kaggle.com/truthr/non-linear-dynamics-trajectories-with-python"
}

Introduction

Natural Language Processing is a fast-advancing field. And it is also one of the fields that require a huge amount of computational resources to make important progress. And although breakthroughs are openly announced, and papers are released in free-to-access repositories such as arXiv, Open Review, Papers with Code, etc., and despite (sometimes) having the code freely available on GitHub, using those language models is not something widely accessible and easy.

Let me provide more context. BERT is a state-of-the-art encoder language model. It takes days to train the model from the ground up even when using very powerful GPUs that only a few entities have access to. In 2019, NVIDIA used 1472 NVIDIA V100 GPUs to train BERT from scratch in 53 minutes. Yes, 1,472!

One estimate puts the cost of training GPT-3, a 175 billion parameter model, for a single training run at $12 Million USD.

And such language models are released every now and then. How do you use these powerful language models for your task?

Here Hugging Face comes to the scene. They aim to solve this problem by providing pre-trained models, and simple API so that you can use them, fine-tune them, and use the API in your applications.

In this article, my goal is to introduce the Hugging Face pipeline API to accomplish very interesting tasks by utilizing powerful pre-trained models present in the models hub of Hugging Face.

To follow through this article, you need not have any prior knowledge of Natural Language Processing. I, however, assume minor prior experience in writing Python code.

In this article, I will go over, describe, and provide examples for the following tasks using Hugging Face pipeline-

• Sentiment Analysis
• Zero-Shot Classification
• Text Generation
• Mask-Filling
• Named Entity Recognition
• Question Answering
• Summarization
• Translation

Installing and Importing

! pip install transformers[sentencepiece] > /dev/null

If you have not seen a pip install with a square bracket in it before, don’t worry. It means that you are installing the transformers package with extra support added for the sentencepiece package. You can find out more about square parentheses installs in pip here.

from transformers import pipeline

We will use the utilities available in the pipeline namespace in the transformers library.

Sentiment Analysis

Before I begin going through the specific pipelines, let me tell you something beforehand that you will find yourself. Hugging Face API is very intuitive. When you want to use a pipeline, you have to instantiate an object, then you pass data to that object to get result. Very simple! You are soon to see what I mean.

classifier_sentiment = pipeline("sentiment-analysis")

That’s it. You call the pipeline() method with the task you want to accomplish as an argument. And you assign a name to it. You are done now. You can now begin to use the object as a function to achieve what you want. Let’s see an example-

See, it returns a dictionary contained in a list that has two items, label and score. The label part tells us its prediction, and the score tells us its confidence score.

As an aside, I think they are structured this way because this structure is easily compatible with .json and similar filetypes which are very common in APIs.

Let’s see another example.

See, how the classifier is aware of my emotion about pineapple pizzas? 🙃

Zero-Shot Classification

When you want to classify something using Deep Learning, in many cases you need to train it with labeled examples. This approach is known as Supervised Learning. Even when leveraging transfer learning, you need to train your model with quite a few labeled examples in the domain of your choice. Zero-shot classification is different than that. In this, you use a pre-trained model to classify a given string and some labels of your choice. The model returns you the confidence score for each model.

Let me show you how this works. First, you instantiate an object and assign it a name just like you did in the case of the Sentiment Analysis pipeline.

classifier_zero_shot = pipeline("zero-shot-classification")

Then you pass a string, along with the labels of your choice to test how well they correspond to your sentence.

See, the given sentence is about a profession, and the model tells you that. And it knows that it is much more related to education than politics.

Essentially, when you are doing Zero-Shot Classification, you are supplying a string to the pipeline, and also labels. The pipeline returns how accurate those labels are.

Text Generation

Before I heard about Deep Learning, I heard about Natural Language Processing, because I heard that you can generate text with it! This is the most exciting part of NLP to me, personally.

As the name suggests, this pipeline lets you generate text. It just needs you to supply a prompt, a text that will determine what the generated text will be. Let me show you how.

generator = pipeline("text-generation")

See what I mean! Even with zero training, it can create logically coherent text that is even interesting to read. Robots will take over writers soon 🤖!

You can control the maximum length of the output of the pipeline, and even ask for multiple possible outputs, like so-

Whoa, it’s poetic!

We can accomplish more interesting things with transfer learning. But that is a story for another day.

Mask Filling

In this pipeline, if there is a word that you hide and supply the string with the hidden word, the pipeline will predict that word. This is like magic!

unmasker = pipeline("fill-mask")

You pass the sentence with the masked word, and it will predict it. You can choose the number of predictions you want to see. You just have to pass a value to the top_k parameter.

It can predict the words quite well. Just remember to mask the word using <mask>.

Named Entity Recognition

This pipeline recognizes the proper nouns in your sentence. And it also classifies them. Let’s see an example.

ner = pipeline("ner", grouped_entities=True)

The grouped_entities parameter just ensures that the pipeline can recognize the names that have more than one word, like, you know, Chocolate Factory.

See how it recognizes a person, his profession, and location on its own, and also labels them as “PER”, “ORG”, and “LOC” (meaning location), respectively.

Question Answering

This pipeline provides you with an answer to a question you ask from the context that you provide.

question_answerer = pipeline("question-answering")

It can answer multiple questions from the same prompt.

Summarization

This is one of the most practical tasks that has existed for a long time. Deep Learning really changes the landscape of the task.

Let’s see it in action.

summarizer = pipeline("summarization")

Let’s summarize a piece of text from a blog post I recently read.

Now, that’s something! The efficacy of the summarizer really shocked me. It does this task so well.

Translation

The translation pipeline translates a piece of text from one language to another. It also lets you choose the language model of your choice.

First, let’s translate a text from English to German. I will use the Helsinki-NLP/opus-mt-en-de model for this task.

translator_de = pipeline("translation", model="Helsinki-NLP/opus-mt-en-de")

Note that you are not explicitly telling the model about the language of the input language or the output language. A language model is only trained for translating from one language to another. It cannot translate other languages. If you want to translate from German to English, you’d have to use another model from the model hub. You can find all the translation models here.

The cool thing is that Hugging Face is not limited to Romance languages or European languages in general. Let me show you by translating a piece of text from Bengali.

translator_bn = pipeline("translation", model="Helsinki-NLP/opus-mt-bn-en")

Conclusion

I have shown you several pipeline APIs and what they do and how they work.

Here are the suggested next steps-

• Experiment. Play with the pipelines by yourself, try different things with different parameters, run inference on examples of your own, test edge cases, run wild with it. I believe that is the best way to learn.
• Do Projects. Use the knowledge you gained here in simple projects. We only learn by doing.
• Hugging Face Course. Hugging Face has recently released a free course where they teach you about Transformers and their library ecosystem. It is a good next step.
• Explore. Explore the Hugging Face website, learn more about their APIs, Services, pre-trained models, and many more things it has to offer.

If you notice any error, mistake, or typo, please feel free to let me know. If something is not clear, please let me know that as well. I will do my best to correct it and will credit you here.

Let me know if you have any questions at all.

(Originally published as a Notebook on Kaggle.)

Introduction

Digital electronics is probably the most impactful and widespread technology now. It forms the basis of all of the chips- from ARM microcontrollers to processors in cellphones, all the way to uberpowerful AMD Threadrippers. In all these cases, digital electronics reign supreme.

It is very useful to learn not only about basic principles but also how these principles and the components of Digital Electronics exactly form the basis of the modern world, i.e., how do we go from basic logic gates to fully functional computers.

Elements of Computing Systems

Digital Electronics is not only important because it is the base of the modern technology that literally governs our lives, but it makes us better as programmers, and computer scientists.

Whether we are self-taught programmers or a person with a 4-year CS degree, we were never taught how we go from basic logic gates to CPUs.

A book that I have come across- The Elements of Computing Systems by By Noam Nisan and Shimon Schocken does just that. It teaches you how the basic building blocks of modern computers actually create the computers.

[Although it shows the new 2nd edition, I am reading the first one]

It promises to teach you, actively, how we can start from nothing but a NAND gate and go all the way to a full-fledged computer with an operating system, and a general-purpose programming language that runs on it, which can be used to create anything. The book is also known as nand2tetris.

I have started reading this book and working through it every Sunday, and I have almost finished a chapter.

Building Gates from Elementary Gates

People who have taken a Digital Architecture class or a Digital Electronics class will know that NAND (NOT AND) gates are called “universal gates” because some combination of them are able to create any other gates. So, when you have a NAND gate with you, you can create a general-purpose computer with it. And the book makes you do that. It makes you create many logic gates starting from AND, NOT, and OR with nothing but a pre-implemented NAND gate.

So, a NAND gate is already implemented and supplied. You first have to create AND, OR, and NOT gates. Then you go on to further create XOR gates, Multiplexors (MUX), Demultiplexors (D-MUX), and multi-bit versions of these gates and basic gates with what you have built previously, viz. AND, OR, NOT, and NAND gates.

This is a fascinating task in itself, and very intellectually stimulating.

This is where the core of the post comes in.

I will show you how to implement any Boolean function with nothing but three logic gates. This is a trick worth knowing.

Let’s start.

Implementing a Boolean Function

// Custom.hdl
// this is a part of Hashnode blog Convoluted demo

CHIP Custom {
    IN a, b, c;
    OUT out;

    PARTS:
    Not(in=a, out=nota);
    Not(in=b, out=notb);
    Not(in=c, out=notc);
    And(a=nota, b=notb, out=notanotb);
    And(a=notanotb, b=notc, out=w1);
    And(a=a, b=notb, out=anotb);
    And(a=anotb, b=notc, out=w2);
    And(a=anotb, b=c, out=w3);
    And(a=a, b=b, out=ab);
    And(a=ab, b=notc, out=w4);
    Or8Way(in[0]=w1, in[1]=w2, in[2]=w3, in[3]=w4, out=out);
}

// Custom.tst
// test file for testing custom chip
// could be found at- https://gist.github.com/ghosh-r/c4e6f5ceb1e7ea2e3ba3601c9de121be

// test file for a custom chip in Convoluted, a Hashnode blog

load Custom.hdl,
output-file Custom.out,
compare-to Custom.cmp,
output-list a%B3.1.3 b%B3.1.3 c%B3.1.3 out%B3.1.3;

set a 0,
set b 0,
set c 0,
eval,
output;

set a 0,
set b 0,
set c 1,
eval,
output;

set a 0,
set b 1,
set c 0,
eval,
output;

set a 0,
set b 1,
set c 1,
eval,
output; 

set a 1,
set b 0,
set c 0,
eval,
output;

set a 1,
set b 0,
set c 1,
eval,
output;

set a 1,
set b 1,
set c 0,
eval,
output;

set a 1,
set b 1,
set c 1,
eval,
output;

// Custom.cmp
// compare file for chip Custom.hdl
// found at- https://gist.github.com/ghosh-r/c4e6f5ceb1e7ea2e3ba3601c9de121be
// test file at- https://gist.github.com/ghosh-r/cef52b9f6ac017e00d64460b025a53fe

|   a   |   b   |   c   |  out  |
|   0   |   0   |   0   |   1   |
|   0   |   0   |   1   |   0   |
|   0   |   1   |   0   |   0   |
|   0   |   1   |   1   |   0   |
|   1   |   0   |   0   |   1   |
|   1   |   0   |   1   |   1   |
|   1   |   1   |   0   |   1   |
|   1   |   1   |   1   |   0   |

// Xor.hdl

CHIP Xor {
    IN a, b;
    OUT out;

    PARTS:
    Not(in=a, out=nota);
    Not(in=b, out=notb);
    And(a=a, b=notb, out=w1);
    And(a=nota, b=b, out=w2);
    Or(a=w1, b=w2, out=out);
}

Conclusion

I showed you how to implement any Boolean function with nothing but three elementary logic gates. However, you should keep in mind that this approach is impractical when there are more variables, and the chip you want is more complicated.

Follow the blog to get more similar posts.

Follow me on Twitter or connect with me on LinkedIn.