FaceGen SDK Manual - Mesh Integration

Overview

Create a statistical shape model (SSM) for all meshes which must change shape with the head. SSMs consist of a base mesh (.fgmesh or .tri) and shape statistics (.egm).

Create a statistical colour model (SCM) for all color maps which contain skin. SCMs consist of a base image (.jpg, .png, etc.), color statistics (.egt file), and a detail texture resampling transform (.fim file) for maps which include the face area.

An SSM and SCMs that are part of the same mesh are referred to as a statistical appearance model (SAM).

SAMs that fit together (such as a head and hairstyles) are referred to as a 'Model Set'

Model sets are described at https://facegen.com/modeller_sets.htm and stored locally here:

~sdk/data/csam

Model sets not included by default in the SDK distribution can be downloaded from the above link.

Head Mesh Design

Having a separate color map for the head (rather than the entire body) is important for performance.

Having the head as a separate mesh entirely (ie separate vertex list) is not necessary.

The best place for the neck seam is at the bottom of the neck (or including a 'bib' below it), so that the color map at the neck seam will be a constant colour and the body color statistics can be small.

The UV map for the head should avoid overlaps, including mirror overlaps.

For integration, the shape of your base head mesh in the face area must conform very closely to the FaceGen internal base face. Get a copy as facegen-internal-base-face.obj with the command:

$> fg3 util internal obj

You can scale, rotate and move this mesh to your coordinate system, but don't change it's shape.

Creating an SSM

If you have an existing mesh which is not closely matched to the shape of the FaceGen internal base face, you can manually align it using 3D modeling tools, or you can use our automated utility to get most of the way there.

Export your highest LOD base head (or body) mesh in neutral expression with no eyelashes or other obsfucating surfaces: 'myShape.obj', then:.

$> fgbl mesh convert myShape.obj myShape.fgmesh
> fg3t nrr myShape.fgmesh name myHead.obj myHead_blink.obj myHeadLOD.obj myHair.obj ...

This will adjust the shape of all the mesh arguments after 'name' to match the FaceGen internal base face.

This command must be done on Windows since you use the GUI (Windows-only) to mark points on the surface for the registration process (points description) If you need to run it again later you can use other platforms as the marked-up myShape.fgmesh is re-saved.

Validate your reshaped base mesh against the FaceGen internal mesh by comparing the outputs 'name_myHead.fgmesh' and 'name_internal.fgmesh'.

In case you later want to use your original face shape within FaceGen it is stored as the face coordinate in 'name.fg'.

If you want to keep the neck seam of your head mesh(es) fixed, edit 'name_myHead.fgmesh' to keep only the seam vertices that you wish to keep fixed and save as 'fixedVerts.obj'

Add any statistical morphs to your models (deltas can be added later):

$> fgbl morph add name_myHead.fgmesh t name_myHead_blink.fgmesh "Blink Eyes"

Generate the EGM files for your meshes. It's best to do this in one go since it takes a long time to run, regardless of how many meshes are being processed:

$> fg3t ssm name_internal.fgmesh -hires -fv fixedVerts.obj name_myHead.fgmesh name_myHeadLOD.fgmesh name_myHair.fgmesh ...

You can add delta morphs to your .fgmesh files after this, but if you add any target morphs (which become statistical morphs once you generate an EGM file), you need to re-run the 'fg3t ssm' command.

Spherical eyes are supported with the 'fg3t ssmEyeC' and 'fg3t ssmEyeI' commands. These will generate SSM data for eye vertices which preserves shape so that the eyes remain spherical for rotation. Not that morphing the eye & lid surfaces will yield better visual results than spherical eyes separate from lids, due to the latter exposing areas where the color cannot be properly reconstructed.

Creating an SCM

For each color map containing skin, you will need to create a statistical color model (SCM) comprising a base image file, an .EGT file and (for the face area only) a .FIM file.

Beginning with your internal base face aligned head mesh (see above), extract all relevent parts for a given color map into a single .OBJ file, for instance 'head.obj', then create the SCM:

$> fg3t scm name_internal.fgmesh head.obj 512

The last number (must be power of 2) gives the pixel size of the base image and statistics file. This can be very small for the body since there is no color variation below the base of the neck. For the head, it's typically 512 if you want to capture the full detail.

This will create the files 'head.png', 'head.egt' and, if the uv map contains any of the face, 'head.fim' and 'head_fade-gen.jpg'. If you plan to integrate the FaceGen face color map with pre-existing body color maps, you should edit 'head_fade-gen.jpg' to ensure proper blending (black maps to FaceGen color and white maps to the body map), then save as 'head_fade.jpg'.

Now view your mesh with the newly created texture image and look carefully at the eyes - you may need to tweak the shape of 'head.obj' and repeat until the eye contour lines up perfectly with the color 'spray painting' process.

Animation Targets

You can use your own bone-based or morph-based targets, or you can copy our morphs (see below).

Since FaceGen shape changes do not affect scale, most animation targets will look appropriate on any face shape, whether done with bones or delta morphs.

For bones, you may have to include bone vertices in the vertex list modified by FaceGen for best results.

For morphs in which the target shape must meet a specific place on the face, such as eye blinks, define your morph as a 'target morph' rather than a 'delta morph'. Most morphs, however, will look best as delta morphs.

To add delta or target morphs, convert your base mesh to a .fgmesh file then create:

$> fgbl mesh convert head.obj head.fgmesh
$> fgbl morph add head.fgmesh d headWithDeltaMorph.obj deltaMorphName
$> fgbl morph add head.fgmesh t headWithTargetMorph.obj targetMorphName

Delta morphs can be added anytime. Target morphs must be added before creating the SSM (below).

Copying FaceGen Animation Morphs

Locate the 'Animate' model set base head mesh:

$> fgbl mesh convert ~sdk/data/csam/Animate/Head/HeadHires.tri base.obj

Create an open pose - a pose where vertices that move differently are well separated, such as 'Phoneme: big aah' and a half blink (of both eyes):

$> fgbl morph list HeadHires.tri
$> fgbl morph apply HeadHires.tri baseOpen.obj d 84 1 t 5 0.5 t 6 0.5

Move/rotate/scale these meshes to align with yours as you did with the internal base mesh and save (eg. 'baseMoved.obj' 'baseOpenMoved.obj'). You will need to adjust the shape of your mesh to ensure that it is closely aligned with 'base.obj' over the face region, and you will also need to create an open pose for your mesh that is closely aligned with 'baseOpen.obj' over the face region. (eg. 'head.obj' 'headOpen.obj').

Generate the similarity transform from our default mesh coordinate system to yours:

$> fgbl mesh xform create transform.txt base.obj baseMoved.obj

Transform our default skin mesh into your coordinate system along with all its morph information:

$> fgbl mesh xform apply transform.txt HeadHires.tri baseMoved.tri

Make a copy of your mesh to receive the morph data:

$> fgbl mesh convert head.obj head.fgmesh

Copy over all the morphs (this will take a couple of minutes per morph):

$> fg3t morphx baseMoved.tri -o baseOpenMoved.obj head.tri -o headOpen.obj