ent-framework
  • Ent Framework
  • Getting Started
    • Code Structure
    • Connect to a Database
    • Create Ent Classes
    • VC: Viewer Context and Principal
    • Ent API: insert*()
    • Built-in Field Types
    • Ent API: load*() by ID
    • N+1 Selects Solution
    • Automatic Batching Examples
    • Ent API: select() by Expression
    • Ent API: loadBy*() Unique Key
    • Ent API: update*()
    • Ent API: deleteOriginal()
    • Ent API: count() by Expression
    • Ent API: exists() by Expression
    • Ent API: selectBy() Unique Key Prefix
    • Ent API: upsert*()
    • Privacy Rules
    • Validators
    • Triggers
    • Custom Field Types
  • Ent API: Configuration and Types
  • Scalability
    • Replication and Automatic Lag Tracking
    • Sharding and Microsharding
    • Sharding Terminology
    • Locating a Shard and ID Format
    • Sharding Low-Level API
    • Shard Affinity and Ent Colocation
    • Inverses and Cross Shard Foreign Keys
    • Shards Rebalancing and pg-microsharding Tool
    • Connection Pooling
  • Advanced
    • Database Migrations and pg-mig Tool
    • Ephemeral (Symbol) Fields
    • Atomic Updates and CAS
    • Custom Field Refactoring
    • VC Flavors
    • Query Cache and VC Caches
    • Loaders and Custom Batching
    • PostgreSQL Specific Features
    • Query Planner Hints
    • Cluster Maintenance Queries
    • Logging and Diagnostic Tools
    • Composite Primary Keys
    • Passwords Rotation
  • Architecture
    • Abstraction Layers
    • Ent Framework, Meta’s TAO, entgo
    • JIT in SQL Queries Batching
    • To JOIN or not to JOIN
Powered by GitBook
On this page

Was this helpful?

Edit on GitHub
  1. Getting Started

Connect to a Database

To start simple, create a PostgreSQL database and several tables there. You can also use you existing database:

$ psql postgresql://postgres:postgres@127.0.0.1/postgres -f ents/cluster.sql
ents/cluster.sql
CREATE TABLE users(
  id bigserial PRIMARY KEY,
  email varchar(256) NOT NULL UNIQUE,
  is_admin boolean NOT NULL DEFAULT FALSE
);

CREATE TABLE topics(
  id bigserial PRIMARY KEY,
  created_at timestamptz NOT NULL,
  updated_at timestamptz NOT NULL,
  slug varchar(64) NOT NULL UNIQUE,
  creator_id bigint NOT NULL,
  subject text DEFAULT NULL
);

CREATE TABLE comments(
  id bigserial PRIMARY KEY,
  created_at timestamptz NOT NULL,
  topic_id bigint REFERENCES topics,
  creator_id bigint NOT NULL,
  message text NOT NULL
);

CREATE TABLE organizations(
  id bigserial PRIMARY KEY,
  name text NOT NULL UNIQUE
);

CREATE TABLE organization_users(
  id bigserial PRIMARY KEY,
  organization_id bigint REFERENCES organizations,
  user_id bigint REFERENCES users,
  UNIQUE (organization_id, user_id)
);

To access that database, create an instance of Cluster:

ents/cluster.ts
import { Cluster } from "ent-framework";
import type { PgClientPoolOptions } from "ent-framework/pg";
import { PgClientPool } from "ent-framework/pg";
import type { PoolConfig } from "pg";

export const cluster = new Cluster<PgClientPool, PgClientPoolOptions>({
  islands: async () => [ // sync or async
    {
      no: 0,
      nodes: [
        {
          name: "island0-master",
          config: {
            connectionString: process.env.DATABASE_URL, // e.g. from .env
            // This object is of the standard node-postgres type PoolConfig.
            // Thus, you can use host, port, user, password, database and other
            // properties instead of connectionString if you want.
            min: 5,
            max: 20,
          } satisfies PoolConfig,
        },
      ],
    },
  ],
  createClient: (node) => new PgClientPool(node),
  loggers: {
    clientQueryLogger: (props) => console.debug(props.msg),
    swallowedErrorLogger: (props) => console.log(props),
  },
});

// Pre-open min number of DB connections.
cluster.prewarm();

Terminology:

  1. Cluster consists of Islands. Each Island is identified by an integer number (there can be many islands for horizontal scaling of the cluster).

  2. Island consists of master + replica nodes (in the above example, we only define one master node and no replicas).

  3. Island also hosts Microshards (in the example above, we will have no microshards, aka just one global shard). Microshards may travel from island to island during shards rebalancing process; the engine tracks this automatically ("shards discovery").

Notice that we define the layout of the cluster using a callback. Ent Framework will call it from time to time to refresh the view of the cluster, so in this callback, you can read the data from some centralized configuration database (new nodes may be added, or empty nodes may be removed with no downtime). This is called "dynamic real-time reconfiguration".

As of prewarm() call, it's explained in Advanced section.

PreviousCode StructureNextCreate Ent Classes

Last updated 1 day ago

Was this helpful?

class accepts several options, one of them is the standard interface. For simplicity, when we define a cluster shape in islands, we just return a list of such configs, to be passed into createClient() lambda.

PgClientPool
node-postgres PoolConfig