For big data problems, you need a big solution. Fortunately, this solution does not have to be restricted to an ArcGIS workspace.
Meet Marco Commander for Pro, the server-side component to Marco Desktop built to assist your team in Spatial Data Discovery and Data Reference Inventory as it pertains to the ArcGIS Pro environment and its unique data types (i.e., ArcGIS Pro Map Projects, Layer Files). It is designed for parallel processing and server-class machines with the sole purpose of being the best Knowledge Management system that your network has ever seen.
IN ONE PLACE
Because Marco Commander for Pro gathers detailed synopses of spatial data, it must include a way to store this information with immediate and secure access. Merging these ideas, the Marco Database was born.
As tools are run, results are stored in a database made up of 30+ tables. Supported databases included SQLite for quick access as well as SQL Server and Oracle for more secure platforms.
With this approach, Database Administrators may open the Marco Database at any point to view results or define locations for additional scans. This resource can then be uploaded in Marco Mystic for full search, filter, and analysis capabilities to better understand the data at hand.
ON A GRAND SCALE
With much of Marco Commander for Pro's functionality geared towards Spatial Data Discovery and Spatial Data Inventory, the tools included in this application allow spatial data details to be gathered that are not likely acquired by common Knowledge Management software.
To name a few, these attributes include:
Full Path to ArcGIS Pro Map Project (.aprx), Layer File (.lyrx), File Geodatabase, Folder, etc.
Date of Creation, Modification, and Last Access
Status (i.e., Moved, Deleted, Crashed, Unknown)
Spatial Reference and Extents of Dataset
Spatial Reference and Extents of Bookmarks and Data Frames
BREAK OUT SPATIAL DATA
A benefit of Knowledge Management software is that it permits not only the discovery of data, but also the opportunity to organize it into more manageable and meaningful resources. One approach to wrangling in this information is the categorization of spatial data, where custom categories may be applied to Spatial Data Discovery results to learn which files match specific terms and keywords.
These matches are output in the form of a series of reports, making it even easier to implement their care in your own management practices.
RECORD & MAP COORDINATES
The Spatial Data Discovery tools within Marco Commander for Pro pull a variety of information from each piece of data it encounters. One such detail is that of extent coordinates for datasets, bookmarks, and data frames alike.
Once recorded in the Marco Database, these can then be visualized within ArcGIS Pro to help determine any datasets not matching current standards - or simply as a matter of interest.
REPORT ON USER
ACTIVITY & OWNERSHIP
While a Knowledge Management system can assist organizations in getting a better grasp on what data it is they have at their disposal, true management of these valuable assets are the users that maintain the quality of each dataset.
Marco Commander for Pro allows Managers and Team Leads insights into how well their teams are accomplishing this by creating reports based on the number of files as well as their health, status, and location credited to each unique user found.
REPOINT BROKEN FILE PATHS
With all of that data, there may be any number of ArcGIS Pro Map Projects containing broken data sources. Since Marco Commander for Pro is run via Command Prompt, resetting these paths inside of a physical ArcGIS workspace is not the most effective use of our time.
Instead, the application employs the same Remapping functionality found in Integrated Marco Desktop that provides users with the opportunity to define data reference connections for all broken files in a central location, known as a Remap File, followed by the automated repointing process against these ArcGIS Pro Map Map Projects and datasets.
REPORT ON DATA & STATISTICS
The reporting capabilities of Marco Commander for Pro do not end at compiling insights on its users. Its diverse range of tools allow your team to generate reports detailing the number of ArcGIS Pro Map Projects, total layers, broken layers, feature classes, raster datasets, and mosaic datasets found at each container and folder connection defined during the Spatial Data Discovery process.
The Integrated Marco Studio suite is essential in helping teams develop and adhere their own data standards. Many organizations ensure distinct metadata elements be attributed to the datasets it owns. The question is, how do we keep track of this when we are responsible for so many files?
By including metadata in those attributes pulled during the Spatial Data Discovery process, we save time rather than reviewing this information by opening each dataset individually. These details can then be more closely inspected to determine how in sync their contents are with preferred Knowledge Management standards.
RECORD HISTORICAL ATTRIBUTES
Is your organization undergoing a data overhaul? Are you making strides to clean-up outdated and neglected files in an effort to return your system to its once spotless condition?
We know how reassuring it is to be able to see what progress has been made - even if we would rather not dwell in the past. Tools within Marco Commander for Pro are designed to store details about each inventory process in a separate location within the Marco Database to serve as a historical marker. Only the most pertinent information, like identification, location, and health, are saved in this way. However, this is essential should you need to see just how far you have come.
DUST OFF THE DATABASE
The Marco Database is responsible for many an important detail it inventories. As it turns out, it also keeps track of those it cannot inventory.
When an error is encountered for an ArcGIS Pro Map Project or dataset, it is recorded in the database. These errors are typically consequences of corrupt documents or files that do not open within the time allotted. Rather than halting processing completely to detail each indiscretion, recording them within the Marco Database allows you to address them in your own time.
While this is helpful, we realize that you do not always need these details as much as you need...say, to know what did work. For this purpose, Marco Commander for Pro also includes tools to perform its own bit of maintenance on the Marco Database by cleaning up error tables on command.
MULTIPROCESSING & MULTITHREADING
We know more is better, and we also know that Marco Commander for Pro would have to agree.
Server-based applications have a unique opportunity not always available to their Desktop counterparts. That is, the chance to do more faster. Marco Commander for Pro embraces this to its fullest, with many of its features employing multiprocessing across servers. The degree to which this is done is fully customizable by the user to fit the network, available time, and amount of data being crawled.