Overview
A client program has their own app or data collection program which results in a spatial output. This could be CSV with an address or lat/lon, a spatial database, or anything in between.
Static output
Client produces ad-hoc updates
If the user is already maintaining their data using ESRI tools then they should be responsible for publishing and maintaining their dataset on EGIS as well.
If the user creates exports from some other system, and that export is in suitable format (eg. CSV, etc.) then that data can be published to EGIS through the web interface.
Update existing content? There is an 'update data' button available on hosted feature services. I could upload a CSV with x,y information and publish as a hosted feature service. I could then manually update the CSV and use that button to update the feature service. This rewrites the entire dataset of the service and doesn't just append new records.
We should note that I tried to write over a tcc_not_geocoded.csv file. This file contains addresses to be geocoded. Updating the CSV content item did not effect the hosted feature service. Updating the feature service resulted in an error.
Error executing tool. Publish Portal Service Job ID: j573ba5fd63274e6f831dc2f19313af3f : ERROR 999999: Something unexpected caused the tool to fail. Please refer to the error help for potential solutions, or contact Esri Technical Support http://esriurl.com/support . Only CSVs with coordinates can be overwritten. Failed to execute (Publish Portal Service). Failed. Failed to execute (Publish Portal Service).
There are formats that we can update via the web, but a CSV that requires geocoding is not one of them.
Client produces regular or frequent updates
If a user produces updates either frequently, or consistently at regular intervals, then it may make sense to automate the publishing of that data to ensure the EGIS content is current.
We'll need to rely on TC network ETL processes to publish this data to EGIS.
Database
TC Hosted Database
Databases hosted in the TC network aren't readable by the EGIS platform in the cloud. This may not be true indefinitely, but until that changes we'll need to rely on TC network ETL processes to publish this data to EGIS.
Cloud hosted database
Databases in the cloud may be readable directly from EGIS. A layer could be described and published from ArcGIS Pro once, and the data should remain current. The client would be responsible for maintaining the metadata of the content item.
TC network ETL processes
Data on the TC network that can't reasonably be manually maintained by its owner will need to rely on scripts or FME workflows to publish to EGIS.
Onboarding At present, the EGIS team has no infrastructure on the TC network, and as such has no capacity to schedule regular data maintenance tasks. We can, however, work with each client individually to help them create ETLs in their infrastructure to keep their content current in EGIS.
Cloud ETL processes
Given the cloud first adoption strategy from TBS, the intent is to create ETL process that can run entirely in the cloud. Data in the cloud should be readable by these ETL processes, which is covered by the User needs 3rd party data use cases.