Fabric Schema

fabric-systems Fabric Schema Reference

// generated_at: 2026/6/26 13:12:18
// iso: 2026-06-26T04:12:18.094Z
// source: data/bigquery/ddl/*.sql (CREATE TABLE only; ALTER reflected in CREATE)

About Fabric

multi-cloud data fabric

Fabric continuously syncs data scattered across e-commerce, stores, customer touchpoints, and analytics, keeping it always flowing and consistent. This document describes the core schema on GCP BigQuery.

Principles

  • Always flowing — batch and near-real-time pipelines that do not stall data
  • Always in sync — aggregate while preserving consistency, not one-way dumps from sources
  • Never down — pipelines designed for multi-cloud resilience
  • Observable — monitor sync success, failures, and lag (e.g. Datadog)

Initial scope (CDP)

Phase 1 builds a Customer Data Platform (CDP) on GCP. Commerce data from Shopify, Smaregi, and similar systems is consolidated in BigQuery so customers, orders, products, and store transactions can be analyzed together.

About this reference

  • Project: fabric-493607
  • Datasets: shopify, smaregi
  • Tables: 6 (auto-generated from DDL)
  • Sources: Shopify Admin API / Smaregi Platform API
  • Load pattern: scheduled upsert via Cloud Run + Scheduler

Data flow (conceptual)

[ Shopify ]     ──→  shopify_orders / shopify_customers / shopify_products / ...
[ Smaregi POS ] ──→  smaregi_transactions
                         │
                         ▼
                  [ GCP BigQuery ]
                         │
                         ▼
              Analytics · CDP · future AWS / multi-cloud

ERD

// inferred from column names (no FK constraints in DDL)

erDiagram SHOPIFY_CUSTOMERS { int customer_id PK string customer_gid string email string first_name string last_name string phone string state boolean verified_email boolean accepts_marketing string tags_json int tags_count int orders_count decimal total_spent string currency string note boolean tax_exempt string tax_exemptions_json int last_order_id string last_order_gid string last_order_name string email_marketing_consent_json string sms_marketing_consent_json string addresses_json int addresses_count string default_address_json timestamp created_at timestamp updated_at timestamp synced_at string payload } SHOPIFY_ORDER_LINE_ITEMS { string tenant_id string shop_id PK string order_id PK int order_numeric_id string order_name timestamp order_created_at int line_no PK string line_item_gid string product_gid int product_id string variant_gid int variant_id string sku string vendor string variant_title string name int quantity int current_quantity int unfulfilled_quantity int refundable_quantity boolean taxable boolean requires_shipping decimal price decimal discounted_unit_price decimal discounted_total string currency string title string line_item_json timestamp synced_at string payload } SHOPIFY_ORDERS { string tenant_id string shop_id string order_id PK int order_numeric_id string customer_gid int customer_numeric_id string customer_email string source_name string channel_handle string order_name int order_number string email string phone string note string currency string presentment_currency decimal total_price decimal subtotal_price decimal total_tax decimal total_discount decimal total_shipping string financial_status string fulfillment_status timestamp cancelled_at string cancel_reason timestamp processed_at timestamp updated_at timestamp created_at boolean closed boolean test boolean taxes_included string discount_codes_json string billing_address_json string shipping_address_json string tags_json int tags_count int line_items_count int total_quantity string line_items_json timestamp synced_at string payload } SHOPIFY_PRODUCTS { int product_id PK string product_gid string title string handle string product_type string vendor string status string tags_json int tags_count string description_html timestamp published_at int total_inventory string first_price string options_json string media_json string price_range_json string featured_image_url int variants_count timestamp created_at timestamp updated_at timestamp synced_at string payload } SHOPIFY_VARIANTS { int product_id PK int variant_id PK string product_gid string variant_gid string product_status string product_title string variant_title string sku string option1 string option2 string option3 string selected_options_json string barcode string price string compare_at_price int inventory_quantity int inventory_item_id string inventory_item_gid string inventory_policy boolean requires_shipping boolean taxable int position timestamp variant_created_at timestamp variant_updated_at timestamp product_created_at string handle string vendor string product_type string variant_image_url timestamp synced_at string payload } SMAREGI_TRANSACTIONS { string tenant_id string contract_id PK string transaction_head_id PK timestamp transaction_date_time timestamp terminal_tran_date_time timestamp enter_date_time timestamp adjustment_date_time timestamp upd_date_time date sum_date string store_id string store_code string terminal_id string terminal_tran_id string staff_id string staff_code string staff_name string customer_id string customer_code string transaction_head_division string cancel_division string sell_division string sum_division string sales_head_division string subtotal string total string tax_include string tax_exclude string amount string return_amount string cost_total string deposit string change string in_tax_sales_total string guest_numbers string transaction_uuid string exchange_ticket_no string tax_rate timestamp synced_at string payload string details } SHOPIFY_CUSTOMERS ||--o{ SHOPIFY_ORDERS : "customer_numeric_id" SHOPIFY_ORDERS ||--o{ SHOPIFY_ORDER_LINE_ITEMS : "order_id" SHOPIFY_PRODUCTS ||--o{ SHOPIFY_ORDER_LINE_ITEMS : "product_id" SHOPIFY_VARIANTS ||--o{ SHOPIFY_ORDER_LINE_ITEMS : "variant_id" SHOPIFY_PRODUCTS ||--o{ SHOPIFY_VARIANTS : "product_id"

// scroll to view full diagram

Schema

shopify.shopify_customers

Shopify Admin GraphQL customers (payload + parse columns at insert)

Column Type NOT NULL PK candidate
customer_idINT64yesPK
customer_gidSTRING
emailSTRING
first_nameSTRING
last_nameSTRING
phoneSTRING
stateSTRING
verified_emailBOOL
accepts_marketingBOOL
tags_jsonSTRING
tags_countINT64
orders_countINT64
total_spentNUMERIC
currencySTRING
noteSTRING
tax_exemptBOOL
tax_exemptions_jsonSTRING
last_order_idINT64
last_order_gidSTRING
last_order_nameSTRING
email_marketing_consent_jsonSTRING
sms_marketing_consent_jsonSTRING
addresses_jsonSTRING
addresses_countINT64
default_address_jsonSTRING
created_atTIMESTAMP
updated_atTIMESTAMP
synced_atTIMESTAMPyes
payloadSTRINGyes

shopify.shopify_order_line_items

Shopify Admin GraphQL order line items (payload + parse columns at insert)

Column Type NOT NULL PK candidate
tenant_idSTRING
shop_idSTRINGyesPK
order_idSTRINGyesPK
order_numeric_idINT64
order_nameSTRING
order_created_atTIMESTAMP
line_noINT64yesPK
line_item_gidSTRING
product_gidSTRING
product_idINT64
variant_gidSTRING
variant_idINT64
skuSTRING
vendorSTRING
variant_titleSTRING
nameSTRING
quantityINT64
current_quantityINT64
unfulfilled_quantityINT64
refundable_quantityINT64
taxableBOOL
requires_shippingBOOL
priceNUMERIC
discounted_unit_priceNUMERIC
discounted_totalNUMERIC
currencySTRING
titleSTRING
line_item_jsonSTRING
synced_atTIMESTAMPyes
payloadSTRINGyes

shopify.shopify_orders

Shopify Admin GraphQL orders (payload + parse columns at insert)

Column Type NOT NULL PK candidate
tenant_idSTRING
shop_idSTRINGyes
order_idSTRINGyesPK
order_numeric_idINT64
customer_gidSTRING
customer_numeric_idINT64
customer_emailSTRING
source_nameSTRING
channel_handleSTRING
order_nameSTRING
order_numberINT64
emailSTRING
phoneSTRING
noteSTRING
currencySTRING
presentment_currencySTRING
total_priceNUMERIC
subtotal_priceNUMERIC
total_taxNUMERIC
total_discountNUMERIC
total_shippingNUMERIC
financial_statusSTRING
fulfillment_statusSTRING
cancelled_atTIMESTAMP
cancel_reasonSTRING
processed_atTIMESTAMP
updated_atTIMESTAMP
created_atTIMESTAMP
closedBOOL
testBOOL
taxes_includedBOOL
discount_codes_jsonSTRING
billing_address_jsonSTRING
shipping_address_jsonSTRING
tags_jsonSTRING
tags_countINT64
line_items_countINT64
total_quantityINT64
line_items_jsonSTRING
synced_atTIMESTAMPyes
payloadSTRINGyes

shopify.shopify_products

Shopify Admin GraphQL products (payload + parse columns at insert)

Column Type NOT NULL PK candidate
product_idINT64yesPK
product_gidSTRING
titleSTRING
handleSTRING
product_typeSTRING
vendorSTRING
statusSTRING
tags_jsonSTRING
tags_countINT64
description_htmlSTRING
published_atTIMESTAMP
total_inventoryINT64
first_priceSTRING
options_jsonSTRING
media_jsonSTRING
price_range_jsonSTRING
featured_image_urlSTRING
variants_countINT64
created_atTIMESTAMP
updated_atTIMESTAMP
synced_atTIMESTAMPyes
payloadSTRINGyes

shopify.shopify_variants

Shopify Admin GraphQL variants (payload + parse columns at insert)

Column Type NOT NULL PK candidate
product_idINT64yesPK
variant_idINT64yesPK
product_gidSTRING
variant_gidSTRING
product_statusSTRING
product_titleSTRING
variant_titleSTRING
skuSTRING
option1STRING
option2STRING
option3STRING
selected_options_jsonSTRING
barcodeSTRING
priceSTRING
compare_at_priceSTRING
inventory_quantityINT64
inventory_item_idINT64
inventory_item_gidSTRING
inventory_policySTRING
requires_shippingBOOL
taxableBOOL
positionINT64
variant_created_atTIMESTAMP
variant_updated_atTIMESTAMP
product_created_atTIMESTAMP
handleSTRING
vendorSTRING
product_typeSTRING
variant_image_urlSTRING
synced_atTIMESTAMPyes
payloadSTRINGyes

smaregi.smaregi_transactions

Smaregi Platform API transactions (upsert by transaction_head_id)

Column Type NOT NULL PK candidate
tenant_idSTRING
contract_idSTRINGyesPK
transaction_head_idSTRINGyesPK
transaction_date_timeTIMESTAMP
terminal_tran_date_timeTIMESTAMP
enter_date_timeTIMESTAMP
adjustment_date_timeTIMESTAMP
upd_date_timeTIMESTAMP
sum_dateDATE
store_idSTRING
store_codeSTRING
terminal_idSTRING
terminal_tran_idSTRING
staff_idSTRING
staff_codeSTRING
staff_nameSTRING
customer_idSTRING
customer_codeSTRING
transaction_head_divisionSTRING
cancel_divisionSTRING
sell_divisionSTRING
sum_divisionSTRING
sales_head_divisionSTRING
subtotalSTRING
totalSTRING
tax_includeSTRING
tax_excludeSTRING
amountSTRING
return_amountSTRING
cost_totalSTRING
depositSTRING
changeSTRING
in_tax_sales_totalSTRING
guest_numbersSTRING
transaction_uuidSTRING
exchange_ticket_noSTRING
tax_rateSTRING
synced_atTIMESTAMPyes
payloadSTRINGyes
detailsSTRING

DDL Sources

// files under data/bigquery/ddl/

  • data/bigquery/ddl/shopify_customers.sql
  • data/bigquery/ddl/shopify_order_line_items.sql
  • data/bigquery/ddl/shopify_orders.sql
  • data/bigquery/ddl/shopify_products.sql
  • data/bigquery/ddl/shopify_variants.sql
  • data/bigquery/ddl/smaregi_transactions.sql