Importing from code and data

TL;DR

Ask your agent

Use Plotwork to draw our database from db/schema.sql. Import it with import_source, then fix anything the diagnostics say was left out.

Use Plotwork to show which roles can call which endpoints in openapi.yaml, as a matrix. Do not guess a cell the spec does not state.

Use Plotwork to draw our local stack from docker-compose.yml and our CI from .github/workflows/ci.yml, and put both pictures in docs/architecture.md.

Use Plotwork to draw production from infra/terraform: send every .tf and .tfvars file under it with import_source, with options.root set to envs/prod.

How an import works

The agent reads the file and sends its text with the format. The importer reads it, writes Plotwork source, and parses it; the tool then validates the source and outlines it, once each.

A folder is sent as files, a list of path and text. A format that spans files (Terraform, Bicep, ARM, Pulumi, CloudFormation, the CDK, SQL, Prisma, Drizzle, Kubernetes, Helm, Kustomize, Backstage, AsyncAPI) reads every file with its endings as one text and passes over the rest with I201; a diagnostic names its file and its line in that file. Other formats read the first file.

Then call draw_diagram on the source to draw it, edit it first, or place it in a document with sync_markdown.

Arrows and frames

An imported arrow names its relationship twice: in its label, and with an icon in the label and a line colour. The same relationship looks the same whichever format it came from.

A line colour is given only where it tells lines apart: when every line of a picture would have the same one, as in an AsyncAPI document where every line is a message, the lines stay plain and the icons say the rest. A legend names each colour used. An arrow with no label, such as a Compose depends_on, has no icon. Structurizr relationships keep their own description and technology, since a C4 line takes no icon.

Every frame has a type, so it has its colour and icon: a Terraform resource drawn round its parts by what it is (a SQL server is data, a Service Bus namespace messaging, a vault security), modules, CDK constructs, and Pulumi components as resource groups, Kubernetes namespaces, Helm releases and charts, Kustomize overlays as environments, GitHub Actions jobs and pipeline stages as CI and CD, and Backstage domains, systems, and owner groups. A Structurizr boundary gets an icon and a tone.

Database schemas

sql reads CREATE TABLE and ALTER TABLE ... ADD from Postgres, MySQL, and SQLite: primary keys, foreign keys inline or at the table level, UNIQUE, and NOT NULL. A column is nullable unless it is NOT NULL or part of the primary key. A migration history can be sent whole: renames, dropped tables and columns, and tables created again are applied in order, so the picture is the schema the last migration leaves. A schema of more than six tables comes back with I204 and ready groups for options.tables (names, or prefixes such as org_*), to draw it in parts that stay readable. A schema of more than 20 tables is drawn whole in parts: an overview of the parts and the foreign keys between them, then the tables of each schema name, cut by the tables that share keys when one is too big. prisma reads schema.prisma models, with each relation labelled by its field. drizzle reads pgTable, sqliteTable, and mysqlTable calls as text; the code is never run.

PictureSourceOutline

API specifications

openapi reads an OpenAPI 3 document in JSON or YAML. The default view, endpoints, is a matrix of each operation against the security schemes, OAuth scopes, and x-roles values the document names. A cell is yes when the operation's security grants it, no when its security is stated and does not, and na when the document does not say: the importer never guesses. A No sign-in column appears when an operation is public through security: []. With options.view set to sequence and options.operationId set to an operation, it draws that one call: the client, the identity provider when OAuth secures it, the API, and each response as a branch.

PictureSourceOutline

With options.view set to map, the endpoints are drawn as a tree: a folder for each tag, in the order the document's tags list gives them, and a row for each endpoint with its summary. The path every endpoint of a tag shares is named once on its folder, so long paths stay readable. An endpoint with several tags is drawn once, under its first. A document whose operations state no security and no x-roles has nothing for the columns of a matrix, so the default view draws the map instead and says so with I203.

A big API is never cut short. A matrix of more than 20 endpoints, or a map of more than 60 rows, is split by tag into several diagrams, returned in parts with I204; whole tags stay together, and a tag too big for one diagram is cut into even pieces titled (1 of 2). options.tags draws only the operations with one of the tags, and options.paths only those under one of the path prefixes, such as /orders; what they leave out is counted in coverage and named with I201.

Only what the pictures need is read: paths, operations, tags, security schemes and requirements, and x-roles. In YAML the bodies of components.schemas and components.examples are skipped before parsing, and a $ref is followed only for the one operation a sequence draws, so a document of several megabytes imports in well under a second on the local server.

Infrastructure as code

terraform reads the .tf, .tf.json, and .tfvars files of a stack, best sent whole in files, with its modules and its environments. Every resource and module block is drawn, nothing is summed into a count, and coverage says Drawn N of N resources:

Types of AWS, Azure, and Google Cloud get their product icons from a table, and any other type of those providers is read from its name, so a new resource type still gets a fitting icon. A resource of another provider, such as random_password, is a plain box. Send the .tfvars files too, and names are drawn as they become: name = "kv-${local.base}" is drawn kv-shop-dev, and a subnet named by var.endpoint_subnet_id is drawn with the name at the end of its id. Variable defaults, terraform.tfvars, and every *.auto.tfvars apply as Terraform applies them; an environment's file, such as env/dev.tfvars, applies when it is the only one sent, or when options.varFiles names it (--var-file on the command line). A variable marked sensitive is never drawn. A variable that is named but never declared frames nothing: the resource sits in the account, with I202 on it. With options.groupBy set to file, resources are framed by the file that declares them instead. When three or more boxes of one frame have the same arrow to one box, such as five jobs pulling images from one registry, they are drawn as one arrow from the frame's own box, images (5). A stack of more than 30 boxes comes back in parts: first an overview with one box for each resource group and one counted arrow for everything between two groups, then one diagram for each group with everything in it. An arrow to a box of another part ends on a dashed box that names the part.

PictureSourceOutline

Terraform in folders

A stack laid out as enterprises keep one, modules/network, modules/data, and envs/dev and envs/prod that call them, is read as Terraform plans it:

options.groupBy set to network nests by network alone, without module frames. A frame of the network module sits in the resource group its resources name, and a database of the data module placed in a subnet of the network module points at that subnet with a subnet arrow.

Terraform plans and state

terraform-plan reads what Terraform itself writes, so values that come from outside the .tf files, such as a registry module's resources or a name a data source returns, are drawn as Terraform resolved them:

Every resource keeps the address the terraform reader gives it, such as module.network.aws_subnet.private["eu-west-1a"], so the overrides of a diagram made from the .tf files carry over to one made from a plan. A plan's configuration gives the arrows, followed through module outputs and the variables a call passes; when a reference names every copy of a block, the known value chooses the copy, so each machine sits in its own subnet. A state has no configuration: a value that holds another resource's id is the arrow, and a dependency no value shows is a dashed depends on arrow, left out when another arrow already leads there. The plan's JSON does not expand local values, so an arrow made through a local is left out with I202. A sensitive value or variable is never drawn.

Bicep and ARM templates

bicep reads a deployment: main.bicep, the module files it uses (sent in files, at their paths), and its .bicepparam values. Every resource is drawn as the Terraform resource that makes the same thing, so Microsoft.KeyVault/vaults is the same Key Vault box, with the same frames and arrow labels, as azurerm_key_vault. A module whose file is sent is a frame in the resource group its scope: names; resources nested in a parent, or with parent:, sit in it; subnets written inline in a network are frames; existing resources say so. Names are drawn as they become, with format('{0}-{1}-api', prefix, env) and '${prefix}-web' resolved and uniqueString() kept by name. A loop over a known list is one box for each item, and a condition the values make false leaves the resource out with I201. A .bicepparam file applies when it is the only one sent, or when options.varFiles names it.

arm reads an ARM template, as az bicep build or the portal writes it, with its parameters file, and draws it as the Bicep reader does: parameters(), variables(), copy loops, condition, dependsOn, and child resources named parent/child.

PictureSourceOutline

Pulumi

pulumi reads three things, and draws each resource as the Terraform resource that makes the same thing, so AWS, Azure, and Google Cloud look the same whichever tool wrote them:

A resource is named by its logical name, or by its URN when two share one. A secret is never drawn. With no component to frame it, the network is a layer of its own: an app or a database placed in a subnet sits beside the network with a subnet arrow, as a network module draws it in Terraform and Bicep.

CloudFormation and SAM

cloudformation reads a template in YAML, with its short tags, or JSON. !Ref, !GetAtt, and !Sub are arrows named by the property that holds them, names are drawn as !Sub, !Join, and parameter defaults make them, a Condition the defaults make false leaves the resource out with I201, and DependsOn is a dashed arrow. A SAM function's API event is an arrow from the API labelled with its route, POST /orders, and a queue or stream event a triggers arrow from its source. A resource is named by its logical id.

AWS CDK

cdk reads the cloud assembly cdk synth writes: send the cdk.out folder (manifest.json, tree.json, each stack's template, and its nested stacks' templates), or one template cdk synth prints. Each resource is read as cloudformation reads it, so a bucket made with the CDK is the same box as one written in CloudFormation or Terraform.

AWS::CDK::Metadata and the asset manifests are passed over, and a stack the manifest names whose template was not sent is I201.

Containers

compose reads a Docker Compose file. Each service is a box whose type and brand mark come from its image, so postgres:16 is a database with the Postgres mark. An arrow goes from a service to each service it depends on, through depends_on, links, or an address in its environment such as postgres://app@db:5432, and a service that shares another's network through network_mode: "service:db" sits in its frame with a shares network arrow. Every network the file declares becomes a frame, and named volumes become storage boxes. Variables are read as Compose reads them with nothing set: ${PORT:-8080} and ${PORT-8080} read as their default, ${FLAG:+on} as nothing, and a variable with no default (${TAG}, $TAG) stays as $TAG, so the box shows the value is set elsewhere. kubernetes reads multi-document manifests the same way: workloads, Ingresses, ConfigMaps, Secrets, and claims become boxes, namespaces become frames, and routes, selectors, addresses, and mounts become arrows.

helm reads what helm template writes and draws it as kubernetes would, framed by the labels Helm adds: the release (app.kubernetes.io/instance), and, in a release of several charts such as an umbrella chart and its subcharts, the chart (helm.sh/chart); app.kubernetes.io/part-of frames the releases of one application. Sent a chart folder instead, it draws the charts (Chart.yaml, and those in charts/) and what each depends on, each arrow labelled with the version Chart.lock resolved, and a dependency values.yaml switches off dashed. The templates themselves are not rendered, and I201 says so.

kustomize reads what kustomize build writes and draws it as kubernetes would, with the hash a generator adds to a ConfigMap's or Secret's name taken off, so the box keeps its name when its data changes. Sent a folder of kustomization.yaml files instead, it draws each overlay, base, and component as a frame of the files, patches, and generators it lists, with arrows for what an overlay builds on and includes, and what each patch patches.

PictureSourceOutline

Software catalogs

backstage reads Backstage catalog descriptors: one catalog-info.yaml of many documents, or every descriptor of a folder. Components, APIs, and resources are boxes typed by what they are (a website is a client, an openapi API carries the OpenAPI mark, a database resource is a database), systems are frames, and domains are frames around their systems. An arrow says a component provides or consumes an API, or depends on another entity, and a subcomponent is a dashed part of. Each box names its owner on its second line; with options.groupBy set to owner, the groups that own the entities are the frames instead, nested by their parent group, with their members named on the tab. Everything is named by its entity reference, such as component:default/orders-api. A Location is not followed, a kind such as Template is left out with I201, and a relation to an entity the files do not describe is I202.

Architecture models

structurizr reads a Structurizr workspace, written in its DSL (workspace.dsl) or exported as JSON, and draws one view of it as a C4 diagram. The view is the first systemContext, container, or component view the workspace defines, or the containers of its first system that has some; options.view (context, container, or component) and options.root (an identifier or a name, such as banking.api) choose another. A view holds what Structurizr's include * holds: the elements it is about and every person, system, or container a relationship joins to them, with a relationship of a component drawn as one of its container at the container level.

autolayout tb or lr of the view sets the direction. An element the view does not reach is I201, with how to draw the view that holds it; a deployment environment is I201, and an !include or workspace extends is named, not fetched.

PictureSourceOutline

Event-driven systems

asyncapi reads AsyncAPI 2.x and 3.x documents in YAML or JSON, one application each. Send one, or every document of a folder in files: channels of one address are one box, so the picture shows where the producers and consumers of an event-driven system meet. Each application is a service, each channel a queue in the frame of its broker, and each operation an arrow labelled with its messages: into the channel for what the application sends, out of it for what it receives.

An operation whose channel is not in its document is I202, a server that holds no drawn channel is I201, and a $ref into another file is read for its name only.

PictureSourceOutline

Pipelines

github-actions reads a workflow file. What starts it (push, pull_request, a schedule, or a manual run) is the first box, and each job follows in the order its needs set. When the steps fit, each job is a frame of its steps; otherwise a job is one box that says how many steps it runs.

PictureSourceOutline

gitlab-ci reads a .gitlab-ci.yml. Its stages are frames down the page in the order of stages (GitLab's .pre, build, test, deploy, .post when it lists none), and its jobs sit inside them. A job with needs is reached from each job it needs, and needs: [] from the start; a job without is reached from every job of the stage before, which is when GitLab starts it. A job that runs by hand, by its rules, or only on some refs is reached by dashed arrows and says so. extends, hidden .jobs, and anchors are resolved and not drawn; parallel and parallel: matrix are counted; a trigger: job points at the child or downstream pipeline it starts. Each include: is named in I201, not fetched.

azure-pipelines reads an azure-pipelines.yml: stages as frames, jobs and deployment jobs inside, a deployment naming its environment, and arrows for dependsOn, a name or a list, at both levels. A stage with no dependsOn waits for the one before it, and dependsOn: [] starts with the pipeline; a condition: makes the arrows into a job dashed. A file of only jobs draws without frames, and a file of only steps as one chain. Templates, in a list or as extends: template:, are named in I201, not fetched.

Neither pipeline is told apart from other YAML by its text: name the file (.gitlab-ci.yml, azure-pipelines.yml) or set format.

Tables of data

csv reads comma, tab, or semicolon separated text with quoted fields, and json reads an array of records, rows with a header, an object of names and numbers, or nested objects of numbers. With options.chart left at auto, the shape of the table chooses the chart. A table too big for one chart is drawn as several in parts, never cut: the places of an XY chart in pages of up to 60 and its series in groups of six, a matrix in pages of 20 rows and 8 columns, a pie of more than 30 slices in pages that each keep one slice for all the others, so every pie is of the whole, flows on several sankeys (a flow back to an earlier node on a sankey of its own, since a sankey only goes forward), and tiles cut along their groups.

Currency signs, thousands separators, and percent signs are read, and the unit goes on the chart.

PictureSourceOutline

Folder listings

tree reads the output of the tree command, a list of paths such as the output of git ls-files, or an indented outline. Files and folders get their icons; an outline of plain words becomes an outline tree. A listing of more than 60 rows, or deeper than six levels, is drawn as several trees in parts, cut along its folders: each part names the path of the folder it is in (src/lib), and no entry is left out.

PictureSourceOutline

Git history

git-log reads git log --oneline --graph --decorate, plain git log, or --oneline. A log with a graph or decorations becomes a git graph: lanes from the graph, branch names from decorations and merge messages, and tags. A plain log with dates becomes a timeline of commits by day, or by month when it spans more than about twelve days. options.view chooses either. A longer history is drawn in pages of up to 40 consecutive commits, newest first, each a git graph of its own; a timeline draws a busy day or month as several periods of five commits, and a long one in pages. No commit is left out.

PictureSourceOutline

Limits

Nothing in a file is left out by a picture's size: past it, the import comes back in parts, each made a little bigger before there are more. An answer holds 24 parts; partsTotal says how many there are and partsPage asks for the next 24. A part can still be dense: validation then warns with W200.

Import codes

Every import diagnostic points into the file you sent.

See also