Skip to main content

Introduction

NucliaDB is the database platform Nuclia uses to store and index data.

Core features:

  • Easily compare the vectors from different models.
  • Store text, files and vectors, labels and annotations.
  • Access and modify your resources efficiently.
  • Perform hybrid, semantic, keyword, fulltext and graph searches.
  • Export your data in a format compatible with most NLP pipelines (HuggingFace datasets, pytorch, etc).

Quick start​

1. Connecting your database to Nuclia Cloud​

Connecting your database to Nuclia Cloud allows you to own your data while utilizing Nuclia's Understanding API™: Get your NUA API Key.

Nuclia's Understanding API™ provides a data extraction, enrichmentment and inference.

By utilizing it, you can allow Nuclia to do all the heavy lifting for you while you own your own data.

2. Install NucliaDB and run it locally​

Assuming you have a PostgreSQL server reachable at this address and mapped to the PG_URL environment variable:

PG_URL=postgresql://postgres:password@HOSTNAME:5432/postgres

You can run NucliaDB locally with docker:

docker pull nuclia/nucliadb:latest
docker run -it -p 8080:8080 \
    -v nucliadb-standalone:/data \
    -e NUA_API_KEY=<YOUR-NUA-API-KEY> \
    -e DRIVER=PG \
    -e DRIVER_PG_URL=$PG_URL \
    nuclia/nucliadb:latest

Or with Python pip:

pip install nucliadb
nucliadb --nua-api-key=<YOUR-NUA-API-KEY> --driver-pg-url=$PG_URL

3. Create your first Knowledge Box​

A Knowledge Box is a data container in NucliaDB.

To help you interact with NucliaDB, install the Python SDK:

pip install nucliadb_sdk

Then with just a few lines of code, you can start filling NucliaDB with data:

from nucliadb_sdk import NucliaDB, Region

sdk = NucliaDB(region=Region.ON_PREM, url="http://localhost:8080/api")

kb = sdk.create_knowledge_box(slug="my_new_kb")

4. Upload data​

You can use it to push unstructured data to Nuclia's processing engine:

import base64

sdk.create_resource(
    kbid=kb.uuid,
    texts={"text": {"body": "I'm Sierra, a very happy dog"}},
    slug="mykey1",
    files={
        "file": {
            "file": {
                "filename": "data.txt",
                "payload": base64.b64encode(b"asd"),
            }
        }
    },
    usermetadata={
        "classifications": [{"labelset": "emotion", "label": "positive"}]
    },
    fieldmetadata=[
        {
            "field": {
                "field": "text",
                "field_type": "text",
            },
            "token": [{"token": "Sierra", "klass": "NAME", "start": 4, "end": 9}],
        }
    ],
)

or you can simply upload a file to Nuclia with curl:

curl "http://localhost:8080/api/v1/kb/<KB_UUID>/upload" \
  -X POST \
  -H "X-NUCLIADB-ROLES: WRITER" \
  -H "X-FILENAME: `echo -n "myfile" | base64`"
  -T /path/to/file

Then insert more data to improve your search index:

sentences = [
    "She's having a terrible day", "what a delighful day",
    "Dog in catalan is gos", "he is heartbroken",
    "He said that the race is quite tough", "love is tough"
]
labels = [
    ("emotion", "negative"),
    ("emotion", "positive"),
    ("emotion", "neutral"),
    ("emotion", "negative"),
    ("emotion", "neutral"),
    ("emotion", "negative")
]
for i in range(len(sentences)):
    sdk.create_resource(
        kbid=kb.uuid,
        texts={"text": {"body": sentences[i]}},
        files={
            "file": {
                "file": {
                    "filename": "data.txt",
                    "payload": base64.b64encode(b"asd"),
                }
            }
        },
        usermetadata={
            "classifications": [{"labelset": labels[i][0], "label": labels[i][1]}]
        },
    )

The extracted data resulting from Nuclia's processing will be pulled by your NucliaDB installation and stored on your local machine.

Now you are ready to perform a search on your data:

results = sdk.find(kbid=kb.uuid, query="To be in love")

or generate answers:

results = sdk.ask(kbid=kb.uuid, query="What is the name of the happiest dog?")

You can also interact via the HTTP api with curl:

curl http://localhost:8080/api/v1/kb/${KB_UUID}/search?query=your+own+query \
  -H "X-NUCLIADB-ROLES: READER"