LLM Nodes and Prompt Templates

System prompt, variables, model settings

An LLM node sends a prompt to a chosen model. You write system and user messages and insert upstream variables into them (the editor offers a variable picker; in the prompt they appear as references such as {{#start.query#}}). Set the model, temperature and maximum tokens per node, and enable structured output when downstream nodes need fields. Keep instructions explicit: tell the model to answer only from provided context when using retrieval, and specify the format and language. Dify's node names, menus and options change between versions; check the current Dify documentation.

Filling a prompt template from variables, run

I ran this with plain Python 3 (scikit-learn 1.9.1 where imported). It models or tests one piece of a Dify app locally; Dify itself was not running. A simplified model of variable insertion: three references are filled from upstream values and one, start.tone, has no value, so it renders empty and is reported. Missing variables silently producing blank text is a common prompt bug.

import re
# Simplified model of inserting upstream variables into a prompt template
variables = {"start.query": "How do I reset my password?", "start.lang": "English",
             "kb.result": "Passwords are reset from Settings > Security > Reset."}
template = ("Answer in {{#start.lang#}} using only the context.\n"
            "Context: {{#kb.result#}}\nQuestion: {{#start.query#}}\nMissing: {{#start.tone#}}")
missing = []
def fill(m):
    key = m.group(1)
    if key not in variables:
        missing.append(key); return ""
    return variables[key]
prompt = re.sub(r"\{\{#([\w.]+)#\}\}", fill, template)
print(prompt)
print("unresolved variables:", missing)

Output:

Answer in English using only the context.
Context: Passwords are reset from Settings > Security > Reset.
Question: How do I reset my password?
Missing: 
unresolved variables: ['start.tone']

Check the rendered prompt in logs

Run logs show the final prompt each LLM node received; look there first when an answer is odd.

Quick check: What happens if a prompt references a variable that has no value?

• The model always refuses
• It can render as empty text, quietly changing the prompt
• The variable is invented automatically
• Dify trains a new model