Hidden Assumptions in graphcast

The system was built around a twelve-hour forecast step on purpose, because the underlying historical data mixes two different kinds of transitions depending on the timing window. By stepping twelve hours it always jumps cleanly between windows. If your data or step timing does not respect this structure, the model is being used outside the regime it was designed for.

What to do: Keep the twelve-hour step and align inputs to the same time-of-day convention the authors used, rather than re-timing steps in ways that break the assimilation-window structure.

The way the authors measured forecast reliability compares the spread of forecasts against a single best-guess past state, not against a spread of plausible past states. They warn that this scoring method unfairly favors forecasts that are too confident at short lead times, and can make a properly-spread forecast look overconfident. So short-range spread numbers should be read with that caveat in mind.

What to do: When judging short-lead-time spread or reliability, remember the paper's caution that the scoring method itself biases these numbers and do not over-interpret apparent over- or under-dispersion early in the forecast.

The model relies on a separate statistics file to scale its inputs and outputs correctly. If that file doesn't exactly match the model you loaded — for example, if you downloaded the wrong statistics file, or mixed files from different model versions — the model will still run without any complaint, but every number it produces will be quietly wrong. The forecast will look like a forecast, but it won't be correct.

What to do: Before running, double-check that your statistics file, your model checkpoint, and your task configuration all come from the same model version — mismatching these is the single easiest way to get plausible-looking but incorrect forecasts.

The model expects your weather data to be formatted in a very specific way: particular variable names, with latitude running from the North Pole down to the South Pole, and specific pressure altitudes. If you get data from any source other than ERA5 — or even ERA5 downloaded through a tool that reorders dimensions — the model will either crash or, more dangerously, silently use the wrong data in the wrong place and produce convincing-looking but wrong forecasts.

What to do: Before feeding in any data, confirm it uses ERA5 variable names, that latitude runs from 90 down to -90, longitude from 0 to 359.75, and that your pressure levels exactly match the list the model was configured with.

The model's internal map between the weather grid and its working graph was built assuming a specific grid resolution. If you feed it data at a different resolution — even a commonly available coarser resolution — the map is rebuilt with the wrong density, and the model silently operates on a completely different structure than it was trained on, producing bad forecasts.

What to do: Always use data at exactly the resolution the checkpoint was trained on; for the standard GraphCast model that means 0.25-degree global ERA5 data.

When you load a saved model file, the code trusts that the file and the software version you're running are perfectly in sync. If the model file was saved with a newer or older version of the code, extra settings can be silently discarded or substituted with wrong defaults, causing the model to run differently than intended with no warning.

What to do: Always use the same version of the code to load a checkpoint as was used to save it, and keep track of which code version produced each checkpoint file.

During multi-step forecasting, the model feeds its own predictions back as inputs for the next step. This only works correctly if the predictions come out in exactly the same format the model expects as input. If anything about the time ordering or variable structure is slightly off, either the model crashes part-way through a long forecast, or all steps after the first use misaligned data with no error shown.

What to do: When customizing inputs or variables, verify that every predicted output variable exactly matches the expected input structure before attempting a multi-step rollout.

The geometric structure of the model's internal graph is built using high-precision math on the CPU, then handed off to the accelerator. On certain hardware setups, that precision is silently cut in half before the model ever sees it, slightly but persistently corrupting the spatial encoding in every single forecast, with no warning.

What to do: If running on TPU or in a restricted-precision environment, verify that spatial edge features are explicitly cast to the intended precision before the first forward pass.

The training loss weights each location on the globe by its true surface area, which requires the latitude values to be in degrees. If a data pipeline inadvertently converts latitudes to radians before they reach the loss function, every location is weighted almost equally, the model gets the wrong training signal, and the resulting model performs worse — with no error or warning during training.

What to do: When writing custom data loaders, make sure latitude coordinates are kept in degrees all the way through to the loss function.

The model always predicts one fixed time-step ahead regardless of what time labels you attach to your output template. If you accidentally ask for outputs at hourly or daily intervals instead of the model's native 6-hour step, the model runs without complaint but the time labels on the outputs are wrong — the forecast data corresponds to different times than indicated.

What to do: Make sure your output template uses time steps that exactly match the model's native forecast interval — 6 hours for GraphCast, 12 hours for GenCast.

Training the model over multiple forecast steps at once requires storing a large amount of intermediate data in memory — one copy for each time step in the rollout. By default, a setting that would cut this memory roughly in half is turned off. Trying to train over many steps without turning it on will simply crash with a confusing out-of-memory error.

What to do: When training with rollouts longer than a few steps, turn on the gradient checkpointing option to avoid running out of accelerator memory.