Materials

A material of the app's own is a .fmat file. The build hook compiles it, the properties give it a name, and SpringSceneMesh draws with it in a scene. The SceneMaterials bean gives one to code.

Why

The engine loads a compiled material by the path of its source, with a function of its own. Without the starter, every app writes the path as a string in its code, waits for the load in every widget that draws with it, sets the parameters on the scene's clock by hand, and finds a wrong path only on the screen. Here the app names the material once, and the starter loads it, times it, says where it is used, and the build says what is wrong.

Example

# lib/resources/application.yml
spring:
  flutter:
    scene:
      materials:
        glass:
          source: assets/materials/glass.fmat
          preload: eager     # not set: as spring.flutter.scene.preload
        neon:
          source: assets/materials/neon.fmat
SceneBackdrop(
  name: 'match-header',
  scene: (context, surface) => [
    SpringSceneMesh(
      material: 'glass',
      geometry: panel,
      parameters: (p) => p
        ..setColor('tint', tint)
        ..setFloat('strength', 0.2),
      everyFrame: (p, elapsed) =>
          p.setFloat('time', elapsed.inMicroseconds / 1e6),
    ),
  ],
  child: header,
)

The file

source is the path of the .fmat file from the app's root, without asset:: the file itself is not an asset and is not read at run time. The app's build hook (buildSceneAssets, see The engine's shaders) compiles every .fmat file under assets/ into lib/spring_gen/flutter_scene_generated/, and the compiled material is what is loaded.

The build fails when a material of the properties is not among the files it compiled:

Materials of the configuration that the build did not compile:
- spring.flutter.scene.materials.glass.source (lib/resources/application.yml) is assets/materials/glas.fmat
The .fmat files under assets/: assets/materials/glass.fmat, assets/materials/neon.fmat.

Eager and lazy

A first load made while the app runs holds back the first frame of its scene: a view waits for its loads. The debug tab says how long it took, and marks a material whose load took longer than a frame then: it wants preload: eager. The time is the load's (the engine reads the compiled programs asynchronously), not a time the page's thread is held.

A material in a scene

SpringSceneMesh draws its geometry with a material of its own: the compiled programs are shared by every mesh of the material, the parameters are the mesh's. It is a scene widget: it goes among the children of a SpringSceneView or a SceneBackdrop (or of the engine's own scene widgets).

Parameter Type Meaning
material String The material's name in spring.flutter.scene.materials
geometry Geometry What is drawn
parameters void Function(MaterialParameters)? Sets the material's parameters on every build
everyFrame void Function(MaterialParameters, Duration)? Sets them on every frame of the scene, with the time it has run, without a build
position, rotation, scale Vector3?, Quaternion?, Vector3? Where it stands, how it is turned, its scale
visible bool Whether it is drawn; true by default
components List<Component> Behaviour on its node (the engine's Component)
children List<Widget> Scene widgets under its node

Materials in code

SceneMaterials (sceneMaterials) gives a material to code:

Member of SceneMaterials Meaning
materials['glass'] The material's SceneMaterialHandle; an unknown name fails naming those there are
names, all, eager The names, every material, the materials loaded with the splash
contains(name) Whether there is a material of that name
Member of SceneMaterialHandle Meaning
create() A new material for one use. The first call loads it, once, and starts the engine when nothing has yet
reload() Loads it again after a failure
state idle, loading, ready or failed (SceneMaterialState)
failure What the first load threw, when it failed
took, loadedWithSplash How long the first load took, and whether with the splash or while the app ran
shading, opaque, parameters How it is lit (unlit, lit, ...), whether it is drawn without blending, the names of its parameters
uses, useCount Its uses on the screen now, by the name of their scene
name, location, eager Its name, its .fmat file, whether it is loaded with the splash

A handle is a ChangeNotifier: it notifies its listeners when its state or its uses change.

The materials are made by the sceneMaterialLoader bean, a SceneMaterialLoader: the engine's loadFmatMaterial (FmatMaterialLoader). A bean of the app's own takes its place, in a test for one.

Precision: what a phone draws differently

The engine opens every material's fragment code with precision mediump float;. A float without a qualifier is then half precision on the GPUs that honor it, and float32 on the ones that ignore it:

Where A mediump float
A desktop or a laptop (any browser, any system) float32: the qualifier is ignored
A phone: every browser of an iPhone (Safari, Chrome, Telegram's), Mali, Adreno half precision: a 10-bit mantissa, the largest value 65504

So a material that computes in pixels, in seconds or with a hash of sin looks right on every desktop and wrong on a phone. The square of a distance of 300 pixels (90000) is already out of the range: a neon glow measured in pixels comes out as one block of light over its whole scene, and noise from fract(sin(x) * 43758.5453) turns into stripes.

One line at the start of the fragment code runs all of the material in float32:

fragment {
  precision highp float;

  void Surface(inout MaterialInputs material) {
    vec2 p = GetUV0();
    // pixels, time, hashes: float32 now
  }
}

Or highp on the variables that need it, and the rest stays at half:

highp vec2 p = GetUV0() * material_params.size;
highp float d = length(p - material_params.center);
The material computes Precision
Texture coordinates from 0 to 1, colours, unit vectors The default is fine
Pixels, sizes of the screen, distances in them highp
Time in seconds (it grows without end) highp
A hash or noise of sin with a large factor highp

The build hook reads each .fmat file and finds what it does; the debug tab shows it under Precision: highp, float32 everywhere, or mediump with the number of places written highp. mediump with none at all is a warning: such a material is right on a phone only if it computes in small numbers alone. A desktop never shows the difference: look at the material on a phone.

When a material fails

A material that fails to load is logged as a warning and published as a SceneFailedEvent (SceneStep.material). The start goes on: an eager material that failed is skipped, and its meshes are not drawn. reload() on its handle tries again.

See also