More

Edit or update ArcGIS Online Web Map data using the REST API

Edit or update ArcGIS Online Web Map data using the REST API


Web Map data is described on the ArcGIS REST API site under the Common data types section. I am curious whether I can request the current JSON configuration as shown in the examples (using /data in the url), modify it, and POST it back? As an example, I could request the web map data, add or modify the bookmarks list (by editing the JSON web map data, see example here), and submit the update to modify the web map item in ArcGIS Online. Does a REST operation or alternative exist for this?


This is definitely possible if the service is a FeatureService. I would do this using Python and either the built-in urllib modules or with requests (my preferred choice).

This example shows how to update features. You can also add features and delete features.

Edit:

You can probably just append a bookmark to the list of bookmarks. You can see this when you view the JSON:

{ "extent": { "spatialReference": {"wkid": 102100}, "xmax": -2050429.2553922953, "xmin": -9946068.529135762, "ymax": 1287382.3514782274, "ymin": -4299247.17182725 }, "name": "(Initial view of Brazil)" },

The Update Item operation is how to do this. See helpful post on GeoNet.

I was not sure how to implement Update Item or what parameter to use to POST the data I was requesting and modifying. I think I was confused between the Items common parameters (such as tags, title, type, etc.) and the actual content (data) of the item (bookmarks, extent, etc.) which is what I wanted to update (by appending bookmark JSON as @crmackey suggested). The solution for me was after requesting and modifying the Item Data, I made a POST request with the modified JSON as the text parameter of the Update Item operation.


ArcGIS Online Feature Service Updates Using FME Best Practice?

Long time lurker first time poster. I'm fairly new to FME (Desktop 2018.1) having recently acquired a license through my organization and I feel like a kid in a candy store. I've developed a few workbenches to auto update our AGOL web maps and ops dashboards with varying levels of success. I'm hoping someone with more experience wouldn't mind sharing their workflows.

Essentially all I'm doing is joining an excel spreadsheet to an SDE feature class (DatabaseJoiner), reconciling it with the existing AGOL feature service (FeatureMerger), tagging insert, update, delete (AttributeCreator), and writing it to AGOL. Very closely following this workflow: https://knowledge.safe.com/articles/19302/how-do-i-write-updates-to-an-existing-arcgis-onlin.html

This seems to work maybe 50% of the time. I'm not getting errors, but sometimes fields are randomly blank. Other times only a subset a features get written. The join works perfect, it's something to do with how it's reconciling with AGOL. I'm working with about 300 features (polygons). Most online posts about this are from like 2015. I'm not even sure if FeatureMerger is the best transformer for this purpose anymore. I apologize if this has already been covered. I tried to search existing posts first.


The arcpy.mapping API only provides access to a limited number of layer properties that can be directly modified but all properties found in the Layer Properties dialog box can be modified using the UpdateLayer function. The information is extracted from a source layer and applied to the layer in a map document that needs to be updated. The source_layer can either be a layer ( .lyr ) file or a layer within a map document. UpdateLayer is a robust function because it can be used in several different ways to produce different results.

One option is to update only a layer's symbology. In this case, the update_layer and source_layer must have similar geometry types. Depending on the renderer (for example, unique value using a particular attribute), the attribute definitions also need to be the same. Updating symbology only is the default behavior.

Another option is to update ALL layer properties, including symbology ( symbology_only=False ). For example, you may want to update the field aliases, selection symbology, query definitions, and so on, for a layer that exists in many map documents. An easy way of doing this is to use ArcMap to modify the layer with the appropriate properties and then save the layer out to a layer ( .lyr ) file. The layer file can then be used as a source_layer to update all the properties for a given layer. Use care when updating all properties and make sure they are not properties that you don't want overwritten.

UpdateLayer is also capable of switching out completely unrelated layers. For example, you could replace a polygon layer with a raster layer or a group layer. This capability is only possible if the symbology_only value is set to False otherwise, you might have conflicts with attributes and/or geometry types. UpdateLayer, when not restricted to symbology, is essentially calling the RemoveLayer and AddLayer functions.

If you want to update the properties for a layer within a layer file, you must modify the properties of the layer in a map document first and then save the changes back to a layer file. See the Layer object's save or saveACopy methods.

If you are only interested in updating a layer's time properties, see UpdateLayerTime.


Setting a refresh interval

In the example above, the earthquake data is refreshed each time the map is opened. However you can set a specific refresh interval for the layer. In this case we know from the information posted on the USGS site that the CSV is updated every five minutes, so we can set the layer to refresh at 5-minute intervals to match.

To set a refresh interval, open the layer options:

And choose Refresh Interval, entering the desired time in minutes:

The layer refresh interval is saved with the map, and updates will occur as specified wherever the map is used (in apps, embedded in websites, etc.).


More about FGDC metadata

If you edit metadata using the FGDC metadata editor add-in, it adds and edits content in the native FGDC metadata XML format inside an item's metadata. This content is displayed under the FGDC Metadata (read-only) heading.

Upgrading existing FGDC metadata content is strongly recommended to support basic operations in the current version of ArcGIS for Desktop that involve metadata.

If you don't upgrade, you can search for items in ArcGIS, and publish them to ArcGIS Online . When information isn't found in the ArcGIS metadata elements associated with the Item Description metadata style, ArcGIS will look for information in the related FGDC metadata elements and index or publish that content instead. In the same manner, you can view an item's simple Item Description summary. However, the rest of the item's FGDC metadata content remains unavailable. Most of the existing content won't appear under the ArcGIS Metadata heading. The buttons in the Description tab won't process any FGDC-format metadata. ArcGIS won't automatically update it with the item's current properties. Other operations in ArcGIS may not find the information they are looking for and require you to retype some information.

If the item's existing FGDC metadata hasn't been upgraded, and you start editing metadata in the Description tab , the Item Description page displays the same content as the simple Item Description display and the Field page shows the existing FGDC attribute descriptions. The other pages will be empty except for any synchronized properties that you can edit. If you edit information on the Item Description page, your edits are saved in both the FGDC metadata elements and the corresponding ArcGIS metadata elements. If you edit information on the Fields page, your edits are saved in the FGDC metadata elements, which are shared with ArcGIS metadata. If you add content using any other page, your changes are saved to the appropriate ArcGIS metadata elements only.

Once an item's metadata includes ArcGIS metadata content that was upgraded or typed by a person, any future indexing and publishing operations only use information stored in the ArcGIS metadata format. Also, the simple Item Description display and the Item Description page in the ArcGIS metadata editor will only show and update ArcGIS metadata elements—the associated FGDC metadata elements will no longer be updated.

When FGDC metadata is upgraded to ArcGIS metadata, a copy of the item's original metadata document is stored within the item's metadata as an enclosure for reference. You can access it using the Metadata Properties button on the ArcCatalog Metadata toolbar.


What’s coming next?

We still have a lot of work ahead of us before our fall release! Here is a preview some of what you can expect to come soon:

  • Smart forms – support for conditional visibility via the Arcade Expression editor, configuring forms on tables, required fields.
  • Android data collection features – snapping, edit multiple, recent value entry
  • Arcade FeatureSet expression support in pop-ups
  • Indoor Mapping support (recognizing indoor map layers and provide a floor picker for switching floors, automatically capture floor information when creating a new feature)
  • More location tracking functionality – including support for rendering tracks on the Android platform, ability to turn tracking on/off from the map, and more.

Thank you once again for your interest in ArcGIS Field Maps and participation in our beta program.


UPDATE: ArcGIS Services will always be discoverable

I have an update to this post I created awhile back. In that post, I stated that (note the bold):

Now, if you think "well I'll just disable the services directory so they can't navigate the tree!", think again. In some silly oversight by ESRI, if you simply append a very simple parameter to the r"/arcgis/rest/services" the folders, map servers, feature servers, suddenly become visible.

I was wrong here. This wasn't a "silly oversight" by ESRI. This was intentional, and neccessary for WebGIS applications to function properly. Let me explain.

For those who don't program, or don't really understand what's happening under the hood of a ArcGIS Web application (or any web application), here's how websites & servers "talk to each other". Almost any modern web application uses a format called "JSON", I'm sure you've heard of it.

Long story short, client side (i.e. the web map loaded on your computer) use this format to send and receive messages with your server, like loading the actual spatial data & style information, etc. This is the only way a website can properly load data from an endpoint. By endpoint I mean something like: https://somesite.com/arcgis/rest/services/Parcels/Mapserver/0

Now what's the point of me telling you all this? Well, this means that ESRI is not going to patch or fix this security loophole, ever. They literally state this here:

They'll receive an error message if they attempt to access the Services Directory once it has been disabled by the admin. All client applications (Web APIs, Desktop, Mobile, etc.) that programmatically access the Services using JSON or other output formats will continue to function as expected.

You need to understand that if you have private, sensitive, or otherwise non-public data you have to set up authentication for that server. You need to keep your public data public, and your private data authenticated.

Now, if you think that itɽ be too hard for someone to traverse your directory tree, or that its too much work to do this: Tools already exist that do this. I know because I've built one myself. I can load up your public map and get every endpoint you have on your server in about 30 seconds. Then I can use one of the many tools available to download all endpoints I'm interested in. That also exists, I made a post about it here. I bring this up especially because I see lots of confusion around this topic, like you can see here

ESRI's services are secure. If you have proper authentication for services, nobody will be able to get your data. But public endpoints are public endpoints.

TLDR: ESRI Isn't going to fix that "?f=json" thing, because its a feature, not a bug. So authenticate your data.

Hi there! I work at Esri and am happy to respond to a few of your questions and statements (and from the comments.

First, I don't argue with any of the specific notes from the post above. What I would like to know is how you would prefer it worked. If you want to have "secure" data - meaning only available to certain people, how would your server know who those people are without an authentication process? Turning off HTML access to REST services is just that, it hides the "helper" page that lets you see the parameters, but our REST API is openly documented (with good reason!) and does not claim to secure your individual services by turning off HTML.

In fact, there are a variety of ways to "secure" content without requiring user authentication:

The first would be to not host your ArcGIS Server on the public Internet, you can host it behind a firewall and allow access only to users in your network, which authenticates you based on network logins and your access to the building.

The second would be to look into using the Esri Resource Proxy (https://github.com/Esri/resource-proxy) which is an application you can run on your web server which can be used to limit which webpages your services can be loaded in. This would effectively block your scraper, while still allowing the services to be seen publicly in https://mysite.com/mymap (if that was an allowed "referer" that you set in the Proxy config). This is another common deployment pattern and lets you have a "publicly-accessible" server/service with access limited to specific website.

Our chosen pattern of using a public REST API is not uncommon, as you can see here: https://github.com/toddmotto/public-apis - there are many, many public REST APIs with similar access methods. In fact, our choice to allow an HTML version of the REST API (what you have disabled in your site) is intended as a feature for users to help developers learn the REST API and discover new services that would interest them. If your services are set up securely, you have nothing to fear from a user simply requesting the same services they can see in your web map. If you want to disable /Query operations, you can do that from Server Manager and only allow Map generation, though I think that's overly cautious for most scenarios.

@clanshonuff What you stated is correct. Federation is the act of transferring the security control over your ArcGIS Server Services to items shared through Portal. If you were to have a secured service in Server, federated with an IWA Portal, an anonymous user could never access that service without being redirected to the Portal (and failing their IWA authentication challenge). This is also as designed. If you want to have all of your federated services require IWA, make sure that all the service items in Portal are not shared to "Everyone" and no one will be able to see them or know that they exist.

I hope this answered a few questions, the summary would be "if you want to secure your data, authenticate your services" which is consistent with every historic version of ArcGIS Server.


Share Power BI reports with maps

Enable your organization to make informed decisions by using accurate maps and location data alongside other crucial business information. Share your Power BI reports—with Esri maps—within your organization by using the Embedded or Publish to web features. Multiple formatting tools enable viewers to customize maps, helping them grasp findings quickly and providing you with more time for analysis.


For ArcMap users

Will my ArcObjects application built for ArcMap work in ArcGIS Pro ?

No, ArcGIS Pro has a new .NET API, and the functionality and workflows are different from what exists in ArcMap . The new API was built using ArcObjects however, ArcObjects is not exposed directly.

Will Python scripts and toolboxes written for ArcMap work in ArcGIS Pro ?

Usually, yes. ArcGIS Pro uses Python 3.x, while other applications in ArcGIS Desktop and ArcGIS Server use Python 2.x. Despite some significant differences between these versions of Python, many geoprocessing scripts can be used as is in both ArcMap and ArcGIS Pro . You can read about Python migration for ArcGIS Pro , and use the Analyze Tools For Pro geoprocessing tool to analyze a script or toolbox for necessary updates.

Are all the geoprocessing tools from ArcMap in ArcGIS Pro ?

The majority of ArcMap geoprocessing tools are available in ArcGIS Pro . However, some tools are not available. In some cases, these tools may be available in a future release. Tools that work with data formats that are not supported in ArcGIS Pro , such as coverages, will not be available in the future. Many new tools are also added to each ArcGIS Pro release which are not added to ArcMap .

Will models built in ArcMap work in ArcGIS Pro ?

Yes, most of the time. However, if the model includes a tool that is not available in ArcGIS Pro , the model will not work. You may be able to remove the tool and use a workaround. Additionally, if a tool in the model has changed in ArcGIS Pro , the model will not work until you update the model (make a backup of the toolbox, open the model in ArcGIS Pro , validate, and save the model). Refer to a tool's help topic to determine what needs to change for the tool to work in ArcGIS Pro . One common issue is that the Calculate Field tool no longer supports Visual Basic (VB) expressions. Learn more about migrating ModelBuilder workflows to ArcGIS Pro .

Can I run models that I build in ArcGIS Pro in ArcMap ?

No, models that are built new, or opened and saved in ArcGIS Pro , cannot be used directly in ArcMap . However, you can save the containing toolbox as a previous version and use that toolbox in ArcMap . It's safest to make a backup of the original toolbox before modifying any models in ArcGIS Pro . Keep in mind that many tools and new data formats are only available in ArcGIS Pro , so the model still may not work in ArcMap .

Can I use my maps from ArcMap in ArcGIS Pro ?

Yes, map documents ( .mxd ), scenes ( .sxd ), and globes ( .3DD ) can be imported into ArcGIS Pro and saved in a project. For more information, see Import a map or a map package in the help topic Add maps and scenes to a project. The same geodatabase and shapefile datasets can be used by both applications. Web layers shared from ArcGIS Pro can be opened in ArcMap . Also, ArcMap and ArcGIS Pro can run side by side on the same machine. However, ArcGIS Pro projects cannot be imported into ArcMap .

I imported a map document with multiple data frames, but I only see one map in ArcGIS Pro . What happened to the other data frames?

When you import a map document ( .mxd ), all the data frames become maps in your ArcGIS Pro project, but only one of them opens. You can open the other maps from the Catalog pane by clicking the Project tab at the top of the pane and expanding the Maps item. Alternatively, you can open the catalog view from the View tab on the ribbon and browse to the maps in your project.

How do I share a map authored in ArcGIS Pro with someone using ArcMap ?

You cannot export a map from ArcGIS Pro as a map document ( .mxd ). However, web maps shared from ArcGIS Pro can be opened in ArcMap , ArcGIS Online , or ArcGIS Enterprise .

Can I use my licensed ArcGIS Desktop extensions with ArcGIS Pro ?

Yes. If you have an ArcGIS Desktop license for an extension product, you are licensed to use the same extension in ArcGIS Pro if it is available. See ArcGIS Pro extensions for a list of available extensions. Extension licenses in ArcGIS Pro are managed using the same license type (for example, Named User) as your ArcGIS Pro license.

Where did my ArcMap styles go?

Many ArcMap styles have been retired and are not included as system styles in ArcGIS Pro . However, these styles can be imported from ArcMap . See Import an ArcMap style into the project. A number of these styles have already been converted to ArcGIS Pro *.stylx format and can be downloaded from ArcGIS Online at Esri Styles.

How do I start an edit session in ArcGIS Pro ?

By default, ArcGIS Pro automatically starts an edit session when you modify existing data or create data. Optionally, you can configure your application settings so that you are explicitly required to start and stop editing. For more information, see Enable and disable editing.

Can I create or use geometric networks in ArcGIS Pro ?

Geometric networks are a read-only dataset in ArcGIS Pro . The capability to manage, model, and analyze simple network systems such as hydrology has been replaced by the trace network. The capability to manage, model, and analyze more advanced network systems for water, gas, electric, telecom, sewer, storm water, and other utilities has been replaced by the utility network.


Services

Connections can be created to ArcGIS Server server sites and to Open Geospatial Consortium, Inc. (OGC) WCS, WFS, WMS, WMTS, and OGC API services.

Connections to ArcGIS Server can be created with user- or publisher-level privileges depending on the URL that is provided and the level of privileges that are allowed for your credentials. Once the connection is established, you can change the connection to have administrator-level privileges, if they are allowed for your credentials.

With ArcGIS Pro , you share web layers using the active portal. If you connect to a server associated with the active portal, you can access the hosted services associated with web layers however, you will have access to the data only. Information determining how to draw the data on a map is defined in the web layer in the active portal and is not available when you access the service directly.

Additionally, map, feature, image, geocode, and geoprocessing services can also be published to a stand-alone ArcGIS Server server site.

The following information relates to services accessed from an ArcGIS Server , WCS, WFS, WMS, WMTS, and OGC API connection:

Direct editing and querying is supported. Layers and tables in a feature service can also be accessed.