Skip to content

Commit

Permalink
Merge pull request #13 from saurabh0719/v0.4.0-develop
Browse files Browse the repository at this point in the history 
v0.4.0 (w checksums), file cleanup for tests.
  • Loading branch information
@saurabh0719
saurabh0719 authored May 14, 2021
2 parents 8536170 + 6aa301f commit 4d46085
Show file tree
Hide file tree
Showing 7 changed files with 179 additions and 100 deletions.
83 changes: 55 additions & 28 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,36 +1,42 @@
<div align="center">
<img src="elara.png" width ="75%">
<p>Elara DB is an easy to use, lightweight NoSQL database written for python that can also be used as a fast in-memory cache for JSON-serializable data. Includes various methods to manipulate data structures in-memory, secure database files and export data.</p>
<p>Elara DB is an easy to use, lightweight NoSQL database written for python that can also be used as a fast in-memory cache for JSON-serializable data. Includes various methods and features to manipulate data structures in-memory, protect database files and export data.</p>
</div>

```sh
$ pip install elara
```

* Latest - `v0.4.0`

Go through the [release notes](#releases) for details on upgrades as breaking changes might happen between version upgrades while we're in beta.

<hr>

## Key Features :
* Offers two modes of execution - normal and secure - exe_secure() generates a key file and encrypts the key-value storage for additional security.
* Offers two modes of execution - normal and secure - exe_secure() generates a key file for additional security.
* Manipulate data structures in-memory.
* Can be used as a fast in-memory cache.
* Choose between manual commits after performing operations in-memory or automatically commit every change into the storage.
* Includes methods to export certain keys from the database or the entire storage.
* Uses python's in-built json module for data serialization.
* Incorporates checksums to verify database file integrity.

## Table of Contents :
* [Installation](#installation)
* [License](#license)
* [Usage](#usage)
* [Basic](#basic)
* [Cache](#cache)
* [API](#api)
* [Fundamentals](#usage)
* [Basic usage](#basic)
* [Cache usage](#cache)
* [Serialization & Storage](#serial)
* [API reference](#api)
* [Strings](#strings)
* [Lists](#lists)
* [Dictionaries](#dict)
* [Update Key](#misc)
* [Export data](#export)
* [Tests](#tests)
* [Releases](#releases)
* [Dependencies](#dependencies)
* [Release notes](#releases)
* [Contributors](#contrib)

<hr>
Expand Down Expand Up @@ -66,12 +72,12 @@ This source code is licensed under the BSD-style license found in the LICENSE fi
```

<span id="usage"></span>
## Usage
## Fundamentals

<span id="basic"></span>
### Basic usage :

You can choose between normally transacting data from the file or you can transact data from an encrypted file.
You can choose between normally transacting data from the database or you can protect your database with a key.

```python
import elara
Expand All @@ -86,9 +92,9 @@ print(db.get("name"))

```

* `exe_secure(db_file_path, commit=False, key_file_path)` - Loads the contents of the encrypted database (using the key file) into the program memory or generates a new key file and/or database file if they don't exist in the given path and it encrypts/decrypts the database file. Data is encoded into `utf-8` and then encrypted using `Fernet encryption`
* `exe_secure(db_file_path, commit=False, key_file_path="edb.key")` - Loads the contents of the encrypted database (using the key file) into the program memory or generates a new key file (default - `edb.key`) if it doesn't exist in the given path and it encrypts/decrypts the database file.

Using `exe_secure()` without a key file or without the correct key file corresponding to the database will result in errors. Key files and DB files can be included inside the `.gitignore` to ensure they're not pushed into an upstream repository.
Using `exe_secure()` without a key file or without the correct key file corresponding to the database will result in errors. Database files are verified with checksums to maintain integrity. Key files and DB files can be included inside the `.gitignore` to ensure they're not pushed into an upstream repository.

* `commit` - this argument defaults to `False` ie. you will have to manually call the `commit()` method to write the in-memory changes into the database. If set to `True`, changes will be written into the file after every operation.

Expand All @@ -105,7 +111,7 @@ print(db.get("num"))
db.commit() # Writes in-memory changes into the file
```

* `exe(db_file_path, commit=False)` - Loads the contents of the database into the program memory or generates a new database file if it doesn't exist in the given path. The database file is NOT encrypted and is present in a human-readable json format.
* `exe(db_file_path, commit=False)` - Loads the contents of the database into the program memory or generates a new database file if it doesn't exist in the given path. The database file is NOT protected and can be accessed without a key.

```python
import elara as elara
Expand Down Expand Up @@ -196,7 +202,10 @@ print(cache.getkeys())

```

Elara uses the `json` module to serialize data, hence it only supports basic python datatypes (`int`, `str`, `dict`, `list` etc.).
<span id="serial"></span>
### Serialization and Storage:

Elara supports basic python datatypes (`int`, `str`, `dict`, `list` etc.).
However, objects (simple and complex) can be stored and retrieved using `get`, `set` and other functions that apply to them as long as they are `in-memory` and `not persisted in the file`, as that would lead to serialization errors.

```python
Expand All @@ -217,12 +226,14 @@ print(cache.get("obj").num)

```

To persist a simple object as a dictionary, use the `__dict__` attribute.
* To persist a simple object as a dictionary, use the `__dict__` attribute.

* Elara uses checksums and a file version flag to verify database file integrity.

<hr>

<span id="api"></span>
## API
## API reference

<span id="strings"></span>
### Strings :
Expand Down Expand Up @@ -334,7 +345,7 @@ db.exportdb('exportdb.txt')
db.exportmem('exportmem.txt')
db.exportkeys('exportkeys.txt', keys = ['one', 'three'])

'''
"""
# exportdb.txt
{
"one": 100,
Expand All @@ -344,44 +355,60 @@ db.exportkeys('exportkeys.txt', keys = ['one', 'three'])
# exportmem.txt
{
"one": 100,
"two": 200,
"three": 300
"three": 300,
"two": 200
}
# exportkeys.txt
{
"one": 100,
"three": 300
}
"""
```

<hr>

<span id="tests"></span>
### Tests :
## Tests

Run this command inside the base directory to execute all tests inside the `test` folder:
```sh
$ python -m unittest -v
```
<hr>

<span id="dependencies"></span>
## Dependencies

- `cryptography`
- `msgpack`

<hr>

<span id="releases"></span>
### Releases :
## Release notes

* Latest - `v0.4.x` (Breaking change)
- `v0.4.0`

`v0.4.x` moves away from the json-based (`dump`, `load`) storage approach used in earlier versions, instead storing it as bytes and has support for checksums and database file version flags for added security.

* Previous - `v0.3.x`

* Latest - `v0.3.0` (`utf-8` encoding)
`v0.3.x` uses `utf-8` encoding while storing data.

`v0.2.1` and earlier used a mix of `ascii` and `base64` encoding. `v0.3.0` uses `utf-8` instead. To safeguard data, its better to export all existing data from any encrypted file before upgrading.
You can then use the `securedb()` method to re-encrypt it.
`v0.2.x` and earlier used a mix of `ascii` and `base64` encoding.

* Previous - `v0.2.1`
To safeguard data, its better to **export all existing data** from any existing database file before upgrading Elara. (use `exportdb(export_path)`).

Donwload the latest release from [here](https://github.com/saurabh0719/elara/releases/).
View Elara's [release history](https://github.com/saurabh0719/elara/releases/).

<hr>

<span id="contrib"></span>
### Contributors :
Author - Saurabh Pujari
## Contributors :
Original author and maintainer - Saurabh Pujari
<br>
Logo design - Jonah Eapen
100 changes: 59 additions & 41 deletions README.rst
View file
Original file line number Diff line number Diff line change
@@ -1,26 +1,29 @@
Elara :
-------
Elara
-----

Elara DB is an easy to use, lightweight NoSQL database written for python that can also be used as a fast in-memory cache for JSON-serializable data. Includes various methods and features to manipulate data structures in-memory, protect database files and export data.

Elara DB is an easy to use, lightweight NoSQL database written for python that can also be used as a fast in-memory cache for JSON-serializable data. Includes various methods to manipulate data structures in-memory, secure database files and export data.
View the `Github repository <https://github.com/saurabh0719/elara>`__.

.. code:: sh
$ pip install elara
Key Features :
--------------
Go through the release notes for details on upgrades as breaking changes might happen between version upgrades while we're in beta.

Key Features
------------

- Offers two modes of execution - normal and secure - exe\_secure()
generates a key file and encrypts the key-value storage for
additional security.
- Offers two modes of execution - normal and secure - exe_secure()
generates a key file for additional security.
- Manipulate data structures in-memory.
- Can be used as a fast in-memory cache.
- Choose between manual commits after performing operations in-memory
or automatically commit every change into the storage.
- Includes methods to export certain keys from the database or the
entire storage.
- Based on python's in-built json module for data serialization.
- Incorporates checksums to verify database file integrity.


Installation
Expand Down Expand Up @@ -51,14 +54,14 @@ License

This source code is licensed under the BSD-style license found in the LICENSE file in the root directory of this source tree.

Usage
-----
Fundamentals
------------

Basic usage :
~~~~~~~~~~~~~

You can choose between normally transacting data from the file or you
can transact data from an encrypted file.
You can choose between normally transacting data from the database
or you can protect your database with a key.

.. code:: python
Expand All @@ -72,16 +75,15 @@ can transact data from an encrypted file.
print(db.get("name"))
# Elara
- ``exe_secure(db_file_path, commit=False, key_file_path)`` - Loads the
- ``exe_secure(db_file_path, commit=False, key_file_path="edb.key")`` - Loads the
contents of the encrypted database (using the key file) into the
program memory or generates a new key file and/or database file if
they don't exist in the given path and it encrypts/decrypts the
database file. Data is encoded into ``utf-8`` and then
encrypted using ``Fernet encryption``
program memory or generates a new key file (default - `edb.key`) if it doesn't exist
in the given path and it encrypts/decrypts the
database file.

Using ``exe_secure()`` without a key file or without the correct key
file corresponding to the database will result in errors. Key files and
DB files can be included inside the ``.gitignore`` to ensure they're not
file corresponding to the database will result in errors. Database files are verified with checksums to maintain integrity.
Key files and DB files can be included inside the ``.gitignore`` to ensure they're not
pushed into an upstream repository.

- ``commit`` - this argument defaults to ``False`` ie. you will
Expand All @@ -104,8 +106,8 @@ pushed into an upstream repository.
- ``exe(db_file_path, commit=False)`` - Loads the contents of the
database into the program memory or generates a new database file if
it doesn't exist in the given path. The database file is NOT
encrypted and is present in a human-readable json format.
it doesn't exist in the given path.
The database file is NOT protected and can be accessed without a key.

.. code:: python
Expand Down Expand Up @@ -208,7 +210,10 @@ file.
print(cache.getkeys())
# ['num1', 'num4', 'num3']
Elara uses the ``json`` module to serialize data, hence it only supports basic python datatypes (`int`, `str`, `dict`, `list` etc.).
Serialization and Storage :
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Elara supports basic python datatypes (`int`, `str`, `dict`, `list` etc.).
However, objects (simple and complex) can be stored and retrieved using `get`, `set` and other functions that apply to them
as long as they are ``in-memory`` and ``not persisted in the file``, as that would lead to serialization errors.

Expand All @@ -229,10 +234,11 @@ as long as they are ``in-memory`` and ``not persisted in the file``, as that wou
print(cache.get("obj").num)
# 19
To persist a simple object as a dictionary, use the ``__dict__`` attribute.
- To persist a simple object as a dictionary, use the ``__dict__`` attribute.
- Elara uses checksums and a file version flag to verify database file integrity.

API
---
API reference
-------------

Strings :
~~~~~~~~~
Expand Down Expand Up @@ -395,7 +401,7 @@ Export data :
db.exportmem('exportmem.txt')
db.exportkeys('exportkeys.txt', keys = ['one', 'three'])
'''
"""
# exportdb.txt
{
"one": 100,
Expand All @@ -405,20 +411,20 @@ Export data :
# exportmem.txt
{
"one": 100,
"two": 200,
"three": 300
"two": 200
}
# exportkeys.txt
{
"one": 100,
"three": 300
}
'''
"""
Tests :
~~~~~~~
Tests
-----

Run this command inside the base directory to execute all tests inside
the ``test`` folder:
Expand All @@ -428,23 +434,35 @@ the ``test`` folder:
$ python -m unittest -v
Releases :
~~~~~~~~~~
Dependencies
------------

- ``cryptography``
- ``msgpack``


- Latest - ``v0.3.0`` (``utf-8`` encoding) :
Releases notes
--------------

- Latest - ``v0.4.0`` :

``v0.4.x`` moves away from the json-based (``dump``, ``load``) storage approach used in earlier versions,
instead storing it as bytes and has support for checksums and database file version flags for added security.

- Previous - ``v0.3.x``

``v0.2.1`` and earlier used a mix of ``ascii`` and ``base64`` encoding. ``v0.3.0`` uses ``utf-8``
instead. To safeguard data, its better to export all existing data from any encrypted file before upgrading.
You can then use the ``securedb()`` method to re-encrypt it.
instead.

- Previous - ``v0.2.1``
To safeguard data, its better to export all existing data from any existing database file before upgrading Elara.
(using ``exportdb(export_path)``)

Donwload the latest release from
View Elara's release history
`here <https://github.com/saurabh0719/elara/releases/>`__.


Contributors :
~~~~~~~~~~~~~~
Contributors
------------

| Author - Saurabh Pujari
| Original author and maintainer - Saurabh Pujari
| Logo design - Jonah Eapen
Loading

0 comments on commit 4d46085

Please sign in to comment.