What is Pasky?

Pasky App tries to simplify the process of loading small amounts of data into Google BigQuery. It functions as an API endpoint to send data. Within the POST, it accepts data in the form of JSON, but also can except other http endpoints to recover data in JSON or CSV format to be loaded. Future development may include supporting XML data.

The idea is, in payload that is sent to Pasky, the data to be loaded is included, along with the GCP project ID, the Dataset, and the Table name, and most importantly, a GCP secret location path containing the credentials of a service account that has BigQuery User permissions on a project.

No other information or planning is required. If the dataset doesn’t exist, it is created automatically. If the table doesn’t exist, it is created automatically with a somewhat accurate table schema. Other parameters can be sent as well, such as to overwrite or append.

Pasky App can also be used to run queries against the tables in BigQuery as well and return the results in JSON format, or write to a new table, but that is not included in this public demo.

Having this allows me to use the same method for all my projects where I want to send data, I can use standard python libraries to fetch data and send it directly to bigquery in one function, and not need to worry about installing a variety of modules in each project.

What is this site?

This site aims to provide an example of how Pasky App Works. You can test the functionality by uploading a csv or json file to see if it can be processed correctly. Also a url can be sent that returns a JSON or CSV file that can be processed.

If successful, the data is loaded into a public BigQuery dataset. All tables expire in 1 day, but please don’t upload confidential information.

Just for fun, the content is then rendered in a datatable for your viewing pleasure and exported if desired.

This is simply a front end to give an example to as to how the API behind Pasky works.

Why did you create Pasky App?

I created Pasky App because I use Google BigQuery for the majority of my database needs. The majority of the time it was only just to load data to be able to visualize it with Looker Studio and make some dashboards. However, every time I had a new idea for a project I would take existing code from the last project I did, and I always found myself changing the code just slightly. Over the years, this resulted in many different projects all using slightly different methods and I wanted to avoid having too many different versions of my code.

Additionally, one of the things that I found the most time consuming was properly defining table schemas, it was annoying and time consuming, and I wanted to simplify this process. I figured that there must be an easy way to load data with a moderately accurate table schema.

Also there is the issue with authentication. Among different GCP projects, and different service accounts, it had become a bit problematic with service account keys being generated and left around in different places, and I wanted to make something just a bit more secure.

What does Pasky mean?

Nothing. In searching for a domain years ago I wanted something like Lampoon, and pasquinade is a synonym of lampoon. I felt like pasquinade was too long or the name of an application so I started calling it pasky for short. It was an afterthought that it sounds like Pass Key, and it sort of functions in that sense. You pass it a key, and data, and it passes the data seamlessly to BigQuery.

Is Pasky App free?

Yes. While I think there is some value of what I have made, I don’t believe it’s worth paying for. I would like to be able to add more functionality and make it more stable, but this is really a hobby project. If others are interested in developing, I can publish the source code.

How can I use Pasky?

Pasky is designed to be able to able to be used as Software as a Service (SaaS). If the application has the possibility of accessing service account credentials with the necessary permissions in a project, the API should function directly.

Want to try, send an email to iwannatry@pasky.dev