Category Archives: Programming

Deploying python function as Lambda with API Gateway

Goal of this exercise is to put our previously developed python function behind a REST API.

Tools/prerequisites we are going to use: Visual Studio Code, NPM, python 3, AWS account, AWS credentials configured locally.

GitRepo is available here.

Api Gateways are important infrastructure elements, as a simple Lambda function is only available from inside AWS ecosystem. REST Api is a common, easy to implement standard to share resource.

To make our Lambda function publicly available we are going to update our previous code base with two things: tell the serverless to create the Api Gateway for our stack and upgrade the response of our utc-time-teller.

Let’s start with the python code changes. As we are going to communicate on HTTP channel JSON is a good model structure to respond with. Edit our handler’s code as following:

import datetime
import json

def handler(event, context) -> str:
    dt = datetime.datetime.now(datetime.timezone.utc)
    utc_time = dt.replace(tzinfo=datetime.timezone.utc)

    return {
        "statusCode": 200,
        "headers": {
            "Content-Type": "application/json"
        },
        "body": json.dumps({
            "utc ": str(utc_time)
        })
    }

We are returning a default HTTP response with statusCode of 200 (everything is OK), telling the receiver that the message is in JSON format (“Content-Type”: “application/json”) and of course adding the body itself.

The Lambda is still testable using the previously shown method:

We can continue with the serverless.yml:

Add these few lines to the end of our function:

    events:
      - http:
          path: /{proxy+}
          method: any
      - http:
          path: /
          method: any

This will generate the API Gateway on AWS. The whole serverless.yml looks like this:

service: BlogDavidPythonLambdaDeploy

provider:
  name: aws
  region: eu-west-1

functions:
  utc-time-teller:
    name: utc-time-teller
    handler: src/lambda_handler.handler
    memorySize: 128
    timeout: 30
    runtime: python3.8
    events:
      - http:
          path: /{proxy+}
          method: any
      - http:
          path: /
          method: any

Now call our deployment script from a command line tool:

serverless deploy

Serverless CLI should add new lines to the logs under the category “endpoints”. Copy the one without {proxy+} ending to the browser and enjoy our newly created API gateway response:

Create, write, read text file using C#

Leave a reply

Goal of this post is to show a quick way to handle text files in C#.

Tool/prerequisites we are going to use: Visual Studio, .NET 6.

GitRepo is available here.

With C# it is pretty straightforward to create, write and read text files:

using System;
using System.IO;

namespace BlogDavid_FileCreateWriteRead
{
    internal class Program
    {
        static void Main(string[] args)
        {
            // Keep info of file in memory
            var fileInfo = new FileInfo("./a.txt");
            // Create the file, then create a stream writer, which is easily used to add text
            using (var fileStream = fileInfo.Create())
            using (var streamWriter = new StreamWriter(fileStream))
            {
                streamWriter.WriteLine("Test data!");
            }
            // StreamReader is a great tool to read text from file
            using (var streamReader = new StreamReader(fileInfo.FullName))
            {
                Console.WriteLine(streamReader.ReadToEnd());
            }

            // Don't let console to close, wait for one char
            Console.ReadKey();
        }
    }
}

Running the above code we will get this result:

Deploying python function to AWS Lambda using serverless

1 Reply

Goal of this exercise is to show how easy it is to deploy our python code to AWS.

Tools/prerequisites we are going to use: Visual Studio Code, NPM, python 3, AWS account, AWS credentials configured locally.

GitRepo is available here.

Serverless is a great tool to have the production code and required cloud computing configuration next to each other. To install serverless, use the following command in your favorite shell:

Note: you will need “npm” installed

npm install serverless -g

Create your folder structure for the project: we are going to need one python file, which will be the host of called function and add a file called serverless.yml, for configuration.

Let’s start with the python file:

import datetime

def handler(event, context) -> str:
    dt = datetime.datetime.now(datetime.timezone.utc)
    utc_time = dt.replace(tzinfo=datetime.timezone.utc)

    return str(utc_time)

It is a pretty simple code, which will return the UTC time formatted to string. Which we need to note is the two parameters of our function: event and context, both of them will be added by AWS. As in this exercise we don’t care about input parameters, just add them to our function.

Go to the serverless.yml and add these lines:

service: BlogDavidPythonLambdaDeploy

provider:
  name: aws
  region: eu-west-1

functions:
  utc-time-teller:
    name: utc-time-teller
    handler: src/lambda_handler.handler
    memorySize: 128
    timeout: 30
    runtime: python3.8

service: is the name of the group.
provider: configuration of cloud computing platform.
name: which cloud computing platform we are using.
region: AWS’ regions are available here.
functions: in the use case of AWS, functions are the Lambdas.
“utc-time-teller”: the identifier of our Lambda.
name: how the function will be named on AWS.
handler: path to our function in the form: path/file.functionName
memorySize: how large lambda we are planning to use. Measured in MB-s. Further details and prices can be found here.
timeout: maximum processing time, measured in seconds.
runtime: container type definition with the needed prerequisites like installed python3.8. Further AWS runtimes can be found here.

If we are done with our files, open up a command line tool and call this command:

serverless deploy

Serverless will pack our code and create the stack on AWS. After it finishes, go to AWS Console and in the search bar look for “lambda”:

Search for the name of our lambda “utc-time-teller”:

Click on it and navigate to the “Test” tab, then hit the “Test” button:

If everything went alright, we can see the result of our Lambda:

Zip files with .NET Core

Leave a reply

GitRepo is available here.

Note: this post is heavily based on this StackOverflow answer.

Zipping files in .NET core is pretty simple and can be done in memory:

public static void ZipMultipleFiles(IEnumerable<FileInfo> files, string zipPath)
        {
            using (var archiveStream = new MemoryStream())
            {
                using (var archive = new ZipArchive(archiveStream, ZipArchiveMode.Create, true))
                {
                    // iterating through files, if doesn't exists, skip that file
                    foreach (var file in files)
                    {
                        if (!file.Exists)
                            continue;

                        // creating place for the file in the zip
                        var zipArchiveEntry = archive.CreateEntry(file.Name, CompressionLevel.Fastest);

                        using (var zipStream = zipArchiveEntry.Open())
                        using (var fileStream = new FileStream(file.FullName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
                            // copying the file's content into the zip
                            fileStream.CopyTo(zipStream);
                    }
                }
                // stream needs to be seeked back to the beginning
                archiveStream.Seek(0, SeekOrigin.Begin);

                // writing the whole zip to a new file
                var zipFile = new FileInfo(zipPath);
                using (var fileStream = zipFile.Create())
                    archiveStream.CopyTo(fileStream);
            }
        }

Console App template from C# version 6

Leave a reply

When you create a new project, starting with C# version 6, you will see the following Program.cs:

// See https://aka.ms/new-console-template for more information
Console.WriteLine("Hello, World!");

It could be scary for some people: no static Main entrypoint, no namespace, no usings. Actually the above template, as mentioned on the commented url above, is translated to this:

using System;

namespace BlogMainTemplate
{
    internal class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
        }
    }
}

If you like the old template more, you can just copy it to your Program.cs without any issues.

How to run python code from Windows console

1 Reply

I am quite new to python programming but happy to use it for quick test cases like API calls or file operations. To run these scenarios I run them in different ways:

From file

One of the easiest way to call your script is to store them in a file and make python run them:

print('I am run from a file!')

content of run-me.py

Run it from command line:

python run-me.py

From inline command line

If you don’t want to save your code to a file, you can run it from inline:

python -c "print('I am from inline command line')"

From inline command line, multi line

Normally python code is segmented by new lines and indentation, but inline we can separate commands by semicolons:

SQLite returning value when deleting

2 Replies

Imagine the scenario when you have an SQLite database and you need to delete some of it’s content based on conditions, then list the IDs what you just deleted.

Normally it would take at least two queries: first select the ids based on the condition, store them in a list, then delete the rows with the just queried ids and continue to work with them.

SQLite has a “RETURNING” keyword, which can come in handy to reduce the queries run to just one.

For example we can utilize it as following:

DELETE FROM [file] WHERE toDelete == 1 RETURNING name

The above code will remove the rows marked “toDelete” and return the name-s of those.

Quick example using python 3:

import sqlite3

con = sqlite3.connect('./test.db')
cur = con.cursor()
cur.execute('CREATE TABLE [file] (id, name, toDelete)')
cur.execute('INSERT INTO [file] (id, name, toDelete) VALUES (1, "testFile1", 0)')
cur.execute('INSERT INTO [file] (id, name, toDelete) VALUES (2, "testFile2", 0)')
cur.execute('INSERT INTO [file] (id, name, toDelete) VALUES (3, "testFile3", 1)')
cur.execute('INSERT INTO [file] (id, name, toDelete) VALUES (4, "testFile4", 1)')   

print(cur.execute('DELETE FROM [file] WHERE toDelete == 1 RETURNING name').fetchall())

con.commit()
con.close()

It will print the following:

[('testFile3',), ('testFile4',)]