dart-setup-ffi-assets

Compiling C Code into Code Assets with Native Assets Hooks

Integrate and automate the compilation and packaging of native C/C++ source code into Code Assets under Dart's overarching Native Assets feature using build and link hooks.

Contents

• Introduction
• Constraints
• Native Interop Packages
• Step-by-Step Workflow
• Choosing an Integration Approach
• Method 1: Local Compilation with Linker Tree-Shaking (Recommended)
• Prerequisite Host Compiler Toolchains
• C Source and Bindings Setup
• Defining the C Library Build Spec
• Implementing hook/build.dart
• Implementing hook/link.dart
• Method 2: Downloading Precompiled Dynamic Libraries
• Why Download Precompiled Binaries?
• Implementing Precompiled Dynamic Downloads
• Verification Checklist
• 1. Local Execution Sandbox
• 2. Verify Target Outputs
• 3. Verify Tree-Shaking Stripping
• 4. Verify Offline Compliance (User Defines)

---

Introduction

Under Dart's Native Assets feature, packages can package native code (like C/C++ libraries) as Code Assets and bundle them automatically during standard development cycles (e.g., dart run, dart test, dart build, and flutter run). The packaging of Code Assets is driven by two programmatic hook scripts placed inside a package's hook/ folder:

1. hook/build.dart: Compiles local C sources to machine code or bundles prebuilt native binaries as code assets for a specific host/target architecture.
2. hook/link.dart: Links built code assets, applying advanced tree-shaking optimizations to strip unused native symbols and compress the runtime binary size.

---

Constraints

[!IMPORTANT] Keep all file resolving platform-independent. Never hardcode absolute target paths, shell scripts, or system command variables. Always use Platform.script.resolve() or Uri-based resolution to ensure scripts are fully portable.

• Hook Locations: Compiling and packaging hooks must reside strictly inside the hook/ directory at the package's root:
• hook/build.dart (Build execution phase)
• hook/link.dart (Optional packaging/linking/tree-shaking phase)
• Compile Toolchain Standard: Use the programmatic APIs from package:native_toolchain_c (e.g. CBuilder and CLibrary) to run compile toolchains. Never invoke raw gcc, clang, or msvc via shell commands.
• Preamble & License Headers: Every handcrafted and generated source file (including bindings, helpers, and hooks) must strictly contain the target package's copyright and licensing header.
• Tree Shaking Mapping: If utilizing compiler tree-shaking, you must map the target Dart method names (e.g. Method.name) back to their raw native C symbol names using a record use mapping generated by FFIgen. The mapping file must reside under lib/src/third_party/ and strictly use the .g.dart extension (e.g., sqlite3.record_use_mapping.g.dart).
• Integrity Safeguards for Precompiled Libraries: If adopting the dynamic download pattern:
• Cryptographic Verification: Downloaded prebuilt binaries must be checked against preconfigured lookup tables containing MD5 or SHA-256 hashes to guarantee binary integrity and prevent tampering.
• Graceful Recovery: Support offline developers by providing fallbacks (such as local compiler execution via flags like local_build).

---

Native Interop Packages

Programmatic build and link hooks for Code Assets leverage three specialized native interop packages:

---

Step-by-Step Workflow

Step 1: Add Dependencies

Add Code Assets hook and toolchain dependencies to your package. You must fetch these dependencies directly from pub.dev.

You can add it automatically using the CLI:

dart pub add code_assets hooks native_toolchain_c record_use dev:ffigen

Or manually declare them in your target package's pubspec.yaml:

dependencies:
  code_assets: ^1.0.0
  hooks: ^0.1.0
  native_toolchain_c: ^0.1.0
  record_use: ^0.6.0

dev_dependencies:
  ffigen: ^20.1.1

Step 2: Define C Specifications

Define your target C library compilation metadata inside lib/src/c_library.dart. This lets both the build and link hooks share a single source of truth for assets, names, and sources.

Step 3: Implement Build and Link Hook Scripts

Write the compilation orchestration script inside hook/build.dart and the dead-code elimination logic inside hook/link.dart.

Step 4: Run the Hook Cycle

Running standard test suites dynamically launches the build and link hook lifecycle in the background:

dart test

---

Choosing an Integration Approach

There are two primary methods for integrating and delivering C/C++ native assets in Dart. Select the one that matches your project requirements:

---

Method 1: Local Compilation with Linker Tree-Shaking (Recommended)

In this approach, the build hook invokes local toolchains (GCC, Clang, MSVC) to compile source files directly. The link hook subsequently filters output symbols utilizing compiler options, retaining only target methods invoked in user code. This represents the standard, robust SQLite pattern under pkgs/code_assets/example/sqlite.

Prerequisite Host Compiler Toolchains

Since package:native_toolchain_c delegates actual dynamic compilation to the host operating system's default toolchain, the development machine must have one of the following compiler packages pre-installed:

• macOS: Xcode Command Line Tools. Install via:

    xcode-select --install

• Linux: GCC or Clang. Install via:

    sudo apt install build-essential

• Windows: MSVC (Microsoft Visual C++). Install the Visual Studio Installer and select the Desktop development with C++ workload.

Note: If no compatible toolchain is discovered on the host path, the build hook script will throw a compilation execution exception. Ensure to specify compiler constraints or adopt Method 2 if toolchains cannot be guaranteed.

C Source and Bindings Setup

Assume a C source defining simple math functions at third_party/sqlite/sqlite3.c with its entry point header at third_party/sqlite/sqlite3.h:

#ifndef SQLITE3_H_
#define SQLITE3_H_

const char *sqlite3_libversion(void);

#endif // SQLITE3_H_

We utilize a programmatic FFIgen script (tool/ffigen.dart) to create FFI bindings in lib/src/third_party/sqlite3.g.dart, enabling recorded usage tracking and producing the lookup metadata map in lib/src/third_party/sqlite3.record_use_mapping.g.dart:

// AUTO-GENERATED FILE - DO NOT MODIFY.
// Generated via ffigen.

const recordUseMapping = {
  'sqlite3_libversion': 'sqlite3_libversion',
};

Defining the C Library Build Spec

Define the centralized library specification in lib/src/c_library.dart:

import 'package:native_toolchain_c/native_toolchain_c.dart';

/// The C build specification for the sqlite library.
final cLibrary = CLibrary(
  name: 'sqlite3',
  assetName: 'src/third_party/sqlite3.g.dart',
  sources: ['third_party/sqlite/sqlite3.c'],
);

Implementing hook/build.dart

Implement hook/build.dart using CLibrary.build. This builds the library to a dynamic library (e.g. .so, .dylib, or .dll) inside the hook's target directory:

import 'package:code_assets/code_assets.dart';
import 'package:hooks/hooks.dart';
import 'package:sqlite/src/c_library.dart';

void main(List<String> args) async {
  await build(args, (input, output) async {
    if (input.config.buildCodeAssets) {
      await cLibrary.build(
        input: input,
        output: output,
        defines: {
          if (input.config.code.targetOS == OS.windows)
            // Ensure C functions are explicitly exported in the Windows DLL
            'SQLITE_API': '__declspec(dllexport)',
        },
      );
    }
  });
}

Implementing hook/link.dart

Implement the link optimization phase in hook/link.dart. This utilizes compiler tree-shaking options (LinkerOptions.treeshake) to compile a minimized, dead-code-eliminated binary based on symbol usage records:

import 'package:hooks/hooks.dart';
import 'package:native_toolchain_c/native_toolchain_c.dart';
import 'package:record_use/record_use.dart';
import 'package:sqlite/src/c_library.dart';
import 'package:sqlite/src/third_party/sqlite3.record_use_mapping.g.dart';

void main(List<String> arguments) async {
  await link(arguments, (input, output) async {
    await cLibrary.link(
      input: input,
      output: output,
      linkerOptions: LinkerOptions.treeshake(
        // Map Dart Method references back to raw C symbol names
        symbolsToKeep: input.recordedUses?.calls.keys.cast<Method>().map(
          (e) => recordUseMapping[e.name]!,
        ),
      ),
    );
  });
}

---

Method 2: Downloading Precompiled Dynamic Libraries

An alternative approach compiles binaries beforehand on a central build machine, archives them, and downloads the target binary during the build hook execution. This matches the paradigm demonstrated in the download_asset hook package.

Why Download Precompiled Binaries?

• Host Constraints: Compiling large C/C++ libraries locally requires a complete compiler setup (GCC, Xcode/SDKs, Visual Studio) that the end-developer's host machine may not possess.
• Compile Speed: Precompiled downloads execute in milliseconds compared to potentially long multi-minute compilation processes.
• Platform Bridging: Allows cross-compiling constraints to be avoided if host architectures are limited.

---

Implementing Precompiled Dynamic Downloads

We configure our build hook to detect local compiler flags (e.g. local_build). If not specified, the hook utilizes HttpClient to pull down platform-specific libraries, calculates the MD5 hash to confirm download safety against a configured hashes lookup table, and registers the binary file as a CodeAsset:

1. Defining Target Hashes (lib/src/hook_helpers/hashes.dart)

Define target MD5 hash checks per platform file in your package sources:

const assetHashes = {
  'libnative_add_macos_arm64.dylib': '4a88f50438a98402db2dbd47b59eb412',
  'libnative_add_linux_x64.so': '9f5e15043aa98402dcdbbd47b59ea520',
  'native_add_windows_x64.dll': 'a881e5043ba98402acdebd47b59fa321',
};

2. Hook Downloader Helper (lib/src/hook_helpers/download.dart)

Implement the downloading and integrity check logic using dynamic target filename matching:

import 'dart:io';
import 'package:code_assets/code_assets.dart';
import 'package:crypto/crypto.dart';

const version = '1.0.0';

Uri downloadUri(String target) => Uri.parse(
  'https://github.com/my-org/my-native-repo/releases/download/$version/$target',
);

Future<File> downloadAsset(
  OS targetOS,
  Architecture targetArchitecture,
  Directory outputDir,
) async {
  final fileName = targetOS.dylibFileName('native_add_${targetOS.name}_${targetArchitecture.name}');
  final uri = downloadUri(fileName);
  
  final client = HttpClient()..findProxy = HttpClient.findProxyFromEnvironment;
  final request = await client.getUrl(uri);
  final response = await request.close();
  
  if (response.statusCode != 200) {
    throw ArgumentError('Download target $uri failed: Code ${response.statusCode}');
  }
  
  final targetFile = File.fromUri(outputDir.uri.resolve(fileName));
  await targetFile.create(recursive: true);
  await response.pipe(targetFile.openWrite());
  
  return targetFile;
}

Future<String> hashAsset(File file) async {
  return md5.convert(await file.readAsBytes()).toString();
}

3. Implementing hook/build.dart

Write the final download build hook incorporating local compilation fallback:

import 'dart:io';
import 'package:code_assets/code_assets.dart';
import 'package:hooks/hooks.dart';
import 'package:my_download_package/src/hook_helpers/hashes.dart';
import 'package:my_download_package/src/hook_helpers/download.dart';
import 'package:native_toolchain_c/native_toolchain_c.dart';

void main(List<String> args) async {
  await build(args, (input, output) async {
    final localBuild = input.userDefines['local_build'] as bool? ?? false;

    if (localBuild) {
      final name = 'native_add_${input.config.code.targetOS.name}_${input.config.code.targetArchitecture.name}';
      final builder = CBuilder.library(
        name: name,
        assetName: 'native_add.dart',
        sources: ['src/native_add.c'],
      );
      await builder.run(input: input, output: output);
    } else {
      final targetOS = input.config.code.targetOS;
      final targetArch = input.config.code.targetArchitecture;
      final outputDir = Directory.fromUri(input.outputDirectory);

      final file = await downloadAsset(targetOS, targetArch, outputDir);

      final fileHash = await hashAsset(file);
      final expectedFileName = targetOS.dylibFileName('native_add_${targetOS.name}_${targetArch.name}');
      final expectedHash = assetHashes[expectedFileName];

      if (fileHash != expectedHash) {
        throw Exception(
          'Security Mismatch: File $expectedFileName hash verification failed! '
          'Found hash: $fileHash, expected: $expectedHash.'
        );
      }

      output.assets.code.add(
        CodeAsset(
          package: input.packageName,
          name: 'native_add.dart',
          linkMode: DynamicLoadingBundled(),
          file: file.uri,
        ),
      );
    }
  });
}

---

Verification Checklist

Before declaring a build or link hook implementation complete, always perform the following checks:

1. Local Execution Sandbox

Run unit tests and confirm the native assets compile/link process completes with no runtime or build tool exceptions:

dart test

2. Verify Target Outputs

Navigate to your package target directory and verify that dynamic binary assets are created for the host system:

• macOS: Verify .dart_tool/resources/ or target directories contain .dylib files.
• Linux: Verify .dart_tool/resources/ or target directories contain .so files.
• Windows: Verify .dart_tool/resources/ or target directories contain .dll files.

3. Verify Tree-Shaking Stripping

To ensure the link hook is actually stripping unused native symbols and compressing binary packaging, perform the following validation:

1. Compile a production bundle of the CLI/app:

   dart build cli bin/main.dart

1. Navigate to the compiled build directory containing the dynamic library.
2. Query the exported dynamic symbol tables:

• macOS:

       nm -gU build/cli/lib/libsqlite3.dylib

• Linux:

       nm -D build/cli/lib/libsqlite3.so

• Windows (using MSVC Developer Command Prompt):

       dumpbin /EXPORTS build\cli\lib\sqlite3.dll

1. Confirm Target Exports: Verify that the command outputs only the explicitly kept entry point functions (e.g. sqlite3_libversion) and does not output any unreferenced/stripped symbols.
2. No Bundle Scenario: If the application does not import or invoke any methods from the native library:

• Verify that the link hook logs: Skipping linking as no symbols are to be kept.
• Verify that no library was built/placed in the production bundle (the .dylib/.so/.dll file is not generated, saving bundle size).

4. Verify Offline Compliance (User Defines)

Confirm offline compliance is fully active and the download fallback executes perfectly offline:

1. Configure the local_build: true define for your package in the package's pubspec.yaml (or the workspace root pubspec.yaml):

   hooks:
     user_defines:
       <your_package_name>:
         local_build: true

1. Disable the machine's network adapter or run in a sandboxed offline shell.
2. Launch unit tests:

   dart test

1. Verify the test suite successfully compiles local source files using host compilers, has no compile errors, and never attempts network download requests.