Troubleshoot Pipelines & Dataflows
Identify and resolve common pipeline and Dataflow Gen2 errors — connection failures, timeout issues, activity errors, and data refresh problems.
Troubleshooting mindset
Think of a detective investigating a crime scene.
You don’t guess — you follow the evidence. In Fabric, the evidence is in the error message, the activity run details, and the Monitoring Hub logs. The pattern is always: What failed? → What was it trying to do? → What does the error say? → What changed recently?
Common pipeline errors
| Error Pattern | Typical Cause | Resolution |
|---|---|---|
| Connection timeout | Source database is unreachable, firewall blocking, VNet misconfiguration | Check network connectivity, verify firewall rules allow Fabric IP ranges, test connection in pipeline settings |
| Authentication failure | Expired credentials, rotated keys, service principal permission removed | Update connection credentials in the pipeline or linked service settings |
| Copy activity: data type mismatch | Source column type doesn't match destination schema | Map column types explicitly in the Copy activity mapping tab, or transform with a Dataflow/notebook first |
| Activity timeout | Activity exceeded its configured timeout (default varies by type) | Increase timeout setting, or optimize the activity (reduce data volume, improve query) |
| Concurrent run conflict | Pipeline is already running and concurrency limit is reached | Wait for the current run, increase max concurrent runs, or investigate why runs overlap |
| Parameter error | Missing required parameter, wrong data type, expression syntax error | Check parameter definitions and expressions — use the expression builder to validate |
Scenario: Carlos's Monday morning pipeline failure
Carlos arrives Monday to find the overnight pipeline failed. He investigates:
- Monitoring Hub: Pipeline “Daily-Production-Load” failed at 1:23 AM
- Activity detail: Copy activity “Copy-SAP-Production” failed
- Error message: “Unable to connect to server ‘sap-db.contoso.com’. Login timeout expired.”
- Recent changes: The infrastructure team rotated database credentials on Friday
Fix: Update the connection credentials in the pipeline’s linked service. Re-run the pipeline.
Common Dataflow Gen2 errors
| Error Pattern | Typical Cause | Resolution |
|---|---|---|
| Refresh failed: data source error | Source connector authentication expired or source unavailable | Re-authenticate the connector in Dataflow settings |
| Evaluation timeout | Query takes too long (complex M query on large dataset) | Enable staging lakehouse for query folding, simplify transformations |
| Mashup engine memory limit | Dataset too large for in-memory processing | Enable staging lakehouse, filter data earlier, or switch to a notebook |
| Schema mismatch on destination | Dataflow output columns don’t match destination table | Update column mapping or evolve destination schema |
| Gateway error | On-premises data gateway is offline or misconfigured | Check gateway status, restart if needed, verify data source settings |
Exam tip: Staging lakehouse solves many Dataflow issues
Many Dataflow Gen2 performance and memory errors trace back to the same root cause: the Power Query mashup engine is trying to process too much data in memory.
Fix pattern: Enable a staging lakehouse. This stages intermediate data as Delta tables in OneLake, enabling the enhanced compute engine to process large datasets efficiently instead of holding everything in the Power Query mashup engine’s memory. If the exam describes a Dataflow Gen2 with memory or timeout errors on large datasets, look for “staging lakehouse” in the answers.
A Copy activity in Carlos's pipeline fails with 'Unable to connect — login timeout expired.' The source is an Azure SQL Database. What should Carlos check first?
A Dataflow Gen2 processing 50 million rows from Salesforce keeps timing out. A staging lakehouse is NOT configured. What is the most likely fix?
Next up: Troubleshoot Notebooks & SQL — resolve Spark job failures, T-SQL errors, and memory issues.