Executable World Models for Robot Manipulation

variables, physical properties and interaction rules as code. It will instantiate these in an existing physics simulator, reusing known principles such as gravity, rigid-body dynamics and collision detection. Inference can therefore focus on the scene-specific structure and physical parameters that must be recovered from observations.

Executable structure provides a direct interface for three capabilities that are difficult to obtain from purely neural video or latent world models:

1. Explainability. The entities, state variables, rules and intermediate states responsible for a prediction can be inspected and tested. Prediction errors can help identify whether a failure comes from perception, an incorrect physical parameter or missing program structure.
2. Intervention. Exposed state variables and rules can be edited to test alternative hypotheses, examine how environmental changes affect behaviour and correct the model.
3. Guardrails. An executable checker can test independently specified limits on exposed physical quantities, such as object speed, and reject violating trajectories before action.

Standard video and latent world models do not provide these interfaces directly. Closed-loop execution also allows persistent prediction errors to trigger targeted model revision. Different tasks require different physical information. For obstacle avoidance, a planner may need only obstacle geometry and motion. Manipulation may also require contact, friction or articulation. Inferring unnecessary detail adds computation and can introduce estimation error. A complete physical model of a complex scene may be neither tractable nor necessary. The aim is therefore not to reconstruct all of its physics, but to identify the physical abstraction sufficient for the task. The task should therefore determine what part of the physical world needs to be modelled. Operational constraints may require additional exposed quantities for guardrail checking. This project will condition model inference on the task and infer only the structure needed for planning. Planning will build on virtual model control, which is an unusually good fit for such stateful models and an expertise of one of the supervisors.

Three connected objectives address this question:

1. Learning executable physical hypotheses. How can an agent infer structured, executable models of physical environments from visual observations and known interactions?
2. Learning task-relevant physical abstractions. How can the task determine what physical structure must be inferred, avoiding unnecessary modelling while retaining what is needed for successful decisions?
3. Decision making with executable beliefs. How can explicit structure and uncertainty be exploited for efficient planning, model revision and independently specified operational constraints?

Student Desirable Background:

A strong background in machine learning is desirable, particularly in areas such as computer vision and robotics. The student should be comfortable with mathematical reasoning and programming. Experience with physics simulation, dynamical systems or control would be useful but is not essential; the project is well suited to a student with strong ML foundations who is interested in learning how structured models of the physical world can support planning and decision making.

By submitting an application for a University of Cambridge project, you consent to your documents being shared with the relevant supervisory team at the partner institution. If you have questions about this process aidecisionscdt@manchester.ac.uk before applying.

How to apply:

Please apply under the University of Manchester application portal and select 'PhD in Artificial Intelligence' (OAA Application form)

When applying, you must include the project’s full name, your supervisor’s name, funding status, academic history, and two referees’ contact details. Applications with missing documents will not be processed. Late or incomplete submissions will not be considered, and the university is not responsible for missed deadlines.

After you have applied you will be asked to upload the following supporting documents:

• Final Transcript and certificates of all awarded university level qualifications
• Interim Transcript of any university level qualifications in progress
• CV
• Supporting statement: A mandatory one- to two-page statement explaining your motivation for postgraduate research, why you chose Manchester, relevant academic or work experience, key research findings, and the techniques or skills you've gained. Applications will be put on hold if this is not provided.
• Contact details for two referees (Please ensure referee email addresses are official university or workplace accounts, as we may need to verify the references.)
• English Language Certificate (if applicable)

Equality, diversity and inclusion is fundamental to the success of The University of Manchester, and is at the heart of all of our activities. We know that diversity strengthens our research community, leading to enhanced research creativity, productivity and quality, and societal and economic impact. (Equality, diversity and inclusion | The University of Manchester)

We actively encourage applicants from diverse career paths and backgrounds and from all sections of the community, regardless of age, disability, ethnicity, gender, gender expression, sexual orientation and transgender status.

We also support applications from those returning from a career break or other roles. We are dedicated to supporting work-life balance and offer flexible working arrangements to accommodate individual needs. Our selection process is free from bias, and we are committed to ensuring fair and equal opportunities for all applicants.

We are dedicated to supporting work-life balance and offer flexible working arrangements to accommodate individual needs, including a part-time study option.

Join us and help grow the UK’s national capability in artificial intelligence, part of the UK’s modern industrial strategy. Everyone is welcome if you are passionate about uncertainty quantification, decision making with humans in the loop and decision making for machine learning systems and their applications in Science and Engineering, and looking to develop your research and innovation skills at the doctoral level to further your career in the AI sector.

Funding Notes

The UKRI AI CDT in Decision Making for Complex Systems is part of the TechFirst offering stipends of £30,000 for students eligible for home fee status. The start date is September 2027.

The Project will be based in University of Cambridge.

References

O'Mahony et al. VDAWorld: World Modelling via VLM-Directed Abstraction and Simulation. SIGGRAPH Asia 2026 https://felixomahony.github.io/vdaworld/
Larby and Forni. Optimal Virtual Model Control for Robotics: Design and Tuning of Passivity-Based Controllers. IEEE Transactions on Robotics 2025 https://arxiv.org/abs/2411.06627