Instituto de Informática, Universidade Federal do Rio Grande do Sul
HTN defines what is valid.
MORL learns what is preferable.
Dynamic preferences determine what matters now.
I want an agent that stays inside the rules it was given and still follows priorities that change while it runs. A hierarchical task network handles the first part: the agent only ever does what the domain allows, so it is valid by construction. The cost is rigidity. When safety starts to matter more than speed, someone has to rewrite the domain. Letting a learner loose on the primitive actions removes the rigidity and throws the guarantee away with it. My MSc moves the learning to the choice of method. NPCs are the test bench: there the agent works with partial information and a deadline per frame, and a player notices as soon as it does something odd.
Filtering: the domain says what is legal
At each compound task, the HTN exposes only the methods whose preconditions hold in the current state. Everything semantically invalid under the constraints written into the domain is gone before learning is consulted at all.
Learning: preference-conditioned MORL picks among the survivors
Each reward is a vector with one entry per objective, so objectives that disagree stay separate and never collapse into one number. Conditioning the policy on a preference vector gives a single agent that covers the trade-off space: for every linear preference, an optimal policy lies in the convex coverage set.
Preferences: what matters, right now
The weight vector carries the current trade-off between mission, safety, resources and time. Changing it changes which method gets selected while the HTN domain stays as written, so adapting to a new priority costs no re-authoring.
With two objectives, progress and survival, and estimated returns of [0.8, 0.2] for one method and [0.5, 0.5] for another, the weights [0.8, 0.2] pick the first. Move them to [0.2, 0.8] and the agent picks the second. The numbers are made up for the example.
Credit where the decision was made
Only the selected method is decomposed, and primitive tasks keep executing conventionally. Credit goes to the hierarchical decision that picked the method. That keeps the learning signal legible, and it lets me explain why the agent did what it did.
How it will be evaluated
Compare
First-applicable HTN
Fixed heuristic HTN
Scalar RL for task–method values
Contextual scalar RL + HTN mask
HTN + preference-conditioned MORL
Stress
Preferences switched at set decisions
Held-out preference vectors
Held-out initial states
Measure
Adaptation after a switch
Generalization
Semantic validity
Decision latency
Materials
The ERAMIA-RS paper is in the proceedings and the PDF is here. The poster was presented at ECOA 2026, in October. Both are a first version, with the formulation and the evaluation protocol. The experiments come next, and the full paper with them in 2027.
Poster presented at ECOA 2026, the IV Escola de Computação Avançada at UFRGS, on 21 and 22 October: architecture, hierarchical credit assignment and the planned experiments.
PrototypeGrid-world prototype with HTN execution, sensing, navigation and replanning. Vector rewards, conflicting objectives and dynamic preferences are next. The repository is private while the work is in progress.