An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
Alright, I got this to run, but it's not pretty. Long story short -- I can't reproduce the successful field mapping via the API source that I can create super-easily by using a text file on the disk as a source.
When using a text file source on the disk with the same json material in it, I can import the schema, and the tool creates the hierarchy and grabs the field names and sets everything up for you to just pick what you want. I'm able to assign the fields as I like, and the mapping understands exactly what the source I'm talking about is assigns it properly. The heart of the problem hasn't been the format of anything. It's been the detail at the very end of the error message -- Source=, There's no API connection source fields attached to the mapping when I create the connections by hand. Fields appear on the screen, but they aren't linked to fields from the Source in the json that's behind it in a proper structure. The json simple doesn't show those mappings at all. The front-end editor isn't working correctly on the mapping page.
So I did this --
I created two versions of the same Copy activity, one that pulls directly from the API (on top) and a second that pulls from a sample output that I copied into a file on my drive (on the bottom). I used that second file-based Source to import schemas, then connected the fields I needed, then tested until it was clean and running correctly. File/Save.
Then I went into the json for the unit and copied the fields part of the code from the file-based activity into the API-based activity --
"mappings": [
{
``"source": {
``"path": "['id']"
},
``"sink": {
``"name": "ID",
``"type": "String"
}
},
{
``"source": {
``"path": "['name']"
},
``"sink": {
``"name": "Agent_Name",
``"type": "String"
}
},
{
``"source": {
``"path": "['title']"
},
``"sink": {
``"name": "Title",
``"type": "String"
}
}
],
After the hand-copy of the source code, I saved the pipe. Closed it. Reopened it. Ran it. Success.
Not pretty, but repeatable, dependable, teachable, verifiable. I've spent enough time ironing all this out and it's not my job to fix the interface's shortcomings. I have a path to solution.
Thank you both! Each of you got me looking at the right things!