Digital Asset Management
for UAV Programs
Learn the best practices to scale your UAV program’s data
management systems effectively.
Learn the best practices to scale your UAV program’s data
management systems effectively.
Commercial and enterprise companies, as well as municipal and government entities have realized the value of UAVs, and thousands of them have moved beyond a simple test unit or pilot program and introduced them into day to day work. Data acquisition, inspection and reconnaissance via drone happens every day, in every state and major city across the US, and even more so across the globe.
As UAV programs deliver direct value to organizations with growing operational cost savings, the programs continue to scale and deploy more frequently. With an increasing number of annual flights, the total amount of data being collected and managed continues to grow. It is essential to any program’s longevity to architect a hardened digital asset management system that can prevent data loss, save time daily when ingesting footage, and make it easy to recall data from an archive built to last for decades.
This eBook explains how to set up and scale a digital asset management program that will reduce pilot error, prevent data loss and save on daily operational time. In particular, we discuss efficient data management structure (file naming, foldering, etc.), establishing asset control, checks and validation; and how to effectively choose a cloud provider for your program. Last, we will provide recommendations for hardware, software and cloud services that will deliver greater operational cost savings to your organization as it scales in the coming years.
Meta Data Details
Authors: Oren Schauble, Will Africano, Eric Davis
# of Pages: 37
Last Update Date: March 2021
Title: Digital Asset Management for UAV Programs Subtitle (one sentence description): Learn the best practices if you want to scale your UAV program’s data management systems effectively. Executive Summary: Commercial and enterprise companies, as well as municipal and other government entities have realized the value of UAVs, and thousands of them have moved beyond a simple test unit or pilot program and introduced them into day-to-day work. Data acquisition, inspection and reconnaissance via drone happen every day, in every state and major cities across the US, and even more extensively around the world. As UAV programs deliver direct value to organizations via growing operational cost savings, the programs continue to scale and deploy more frequently. With an increasing number of annual flights, the total amount of data being collected and managed continues to grow. It is essential to any program’s longevity to architect a hardened digital asset management system that can prevent data loss, save time daily when ingesting footage, and make it easy to recall data from an archive built to last for decades. This ebook explains how to set up and scale a digital asset management program that will reduce pilot error, prevent data loss, and save on daily operational time. In particular, we discuss efficient data management structure (file naming, foldering, etc.); establishing asset control, checks and validation; and how to choose an effective cloud provider for your program. Lastly, we will provide recommendations for hardware, software and cloud services that will deliver greater operational cost savings to your organization as it scales in the coming years. 1. Introduction - Current state of Commercial Drone programs We’ve reached a new era in the commercial drone industry. Commercial and enterprise companies, as well as municipal and government entities have realized the value of UAVs, and thousands of them have moved beyond a simple test unit or pilot program, and actually introduced them into day-to-day operations. Data acquisition, inspection and reconnaissance via drone happen every day, in every state and in major cities across the US, and even more extensively around the world. This isn’t a level of widespread adoption in which drones appear noticeably in the skies making everyday life seem different, but it is a step in that direction. Now the challenge in the industry has moved from asking organizations to deal with the various regulatory hurdles and begin adopting the technology to integrating it into workflows and scaling. This is a part of the adoption curve where technology, education and communication become paramount, and when real change can happen as best practices, technology stacks and thought leaders push drone adoption from novelty to normal. Here are a few of the scenarios facing this era of drone technology programs: Pilot programs have become part of workflows and are now constrained Now, small programs bet on by future-forward members of organizations are actually asked for and relied on by key parts of organizations. But many of these pilot programs remain just that: in pilot program format. A few people, a key initiative, and a few sets of hardware are now becoming constrained resources as more parts of the organization are pushing to utilize their value. Today’s internal champions aren’t advocating for budget to determine potential value, or for a chance to prove their drone program in the field. Instead they’re advocating for the budget and personnel resources required to expand their programs, and for the process buy-in to fundamentally change how tasks get executed. Expansion is challenged by technical debt Because of the fast-moving nature of the drone industry, and the very trial-and-error format of finding software and workflow solutions its champions have been forced to figure out to make their programs work--there are inherent challenges to scaling programs. Teams are often operating with mismatched technology -- different drones from different eras, sometimes more than one software solution, checklists and apps utilizing them differently, etc. Teams are expanding with new tech while other teams use older solutions -- and often a piece of software or use of that antiquated software is still the driving force behind a program’s operation. Wondering about the next generation of drone hardware The skies are still dominated by DJI. But something significant has happened in the last 2-3 years -- DJI’s mid-level drones have not really advanced or innovated significantly. After a period of extremely rapid innovation, when it seemed like the previous generation drone was outdated within six months, we haven’t seen a significant improvement since the Mavic 2 or Phantom 4 in the category of workhorse drones utilized by the majority of programs. And honestly, we may not. Those are already excellently priced products, packed with technology, and frankly, DJI has no competition in this space that is worth mentioning on a global scale. If your program has been waiting for what that next generation of hardware from DJI will be – be advised that it might not be coming, at least not soon; and if it does, it will be a minor shift at best. Where there are exceptions to that rule are in industries that care about the integrity of their data and buying American first. This is specific to US infrastructure companies, some public safety orgs, and government institutions. They’re now presented with a slate of drone options, approved by the US government and made in America, led by companies like Teal, Skydio and Vantage Robotics. These options are expensive, but powerful, and blaze a new path forward for developing a drone program--hand in hand with small, US-focused teams who will build exactly the hardware and software their clients ask for as they look to gain a foothold in the global drone market, and someday scale to scenarios in which the economics make sense for them to compete with DJI in specific niche areas. What we’re building in this ebook Moving forward, drone programs need to wipe the slate clean from their trial-and-error periods of growth. Teams must build a process for integrating new technology and process updates rapidly, and leaders need to bite the bullet and build truly scalable workflows, all with buy-in throughout their organizations. In partnership with GNARBOX – the leading provider of field backup solutions and digital asset management – we’ve spoken with teams in the field and experts in the industry to help showcase some specific workflows and technology opportunities that can help program leaders and their teams tackle this next set of challenges. Section 1 - The Issues facing PMs today – a List with descriptions Today’s project managers are getting into the nitty gritty of scaling programs based on extensive data collection and management. Below is a recap of the top challenges program managers need to be considering. Consistent asset management - With multiple pilots, drone systems and data types being captured, there is plenty of room for asset management to fall flat at multiple points in the chain. At the end of the day, data control is people control, and implementing a strong process, explaining the reasoning behind that process to your team, and following up on ensuring the process is followed are the keys to success. Risk reduction - Now that drone programs have established value, risk reduction in data management is focused on how teams can make sure that value is realized consistently, with no exceptions. Losing a dataset, incorrect data capture at a location, or lack of backups could poke holes immediately in an emerging program’s credibility. Vetting technology without a clear reference source - Right now, there is no set of home bases on the web for discussing the industry standards for hardware and software choices in drones. At best there are user forums, and many programs simply don’t have the tools other successful programs use. Until these industry resources exist, communication with key stakeholders at other companies undergoing the same processes is the only way to vet systems choices and process recommendations. This can lead to sunk costs in improper systems, and technical debt that slows down program growth. Complex technology choices - Drone programs often touch everything from hardware to software to server infrastructure, storage solutions and various realms of IT. Expecting drone program leaders to understand and build around existing tech or help make choices for considerable tech infrastructure is a big ask, and close coordination with IT programs that may be antiquated creates challenges for effective scaling. Documenting process in a rapidly changing industry - Drone teams are new, and consistently adding new members. In addition, the process often needs to change with every new tech generation or aircraft model. Keeping up with training, process changes and rollout is often a role of its own that’s not being considered by many programs. Section 2 - Establishing Successful Drone programs - Overview of Mentalities and Strategies There are many drone programs that exist in limbo with light adoption and moderate success, but without quite getting the footing they need to expand. Then there are programs that are thriving--finding value, getting adopted across other parts of their organization, and really growing. We’ve found that when we speak to people in the latter programs, there is a specific combination of mentalities (philosophies that drive actions) and strategies (tactical approaches throughout the day to day) that are almost always in play. Mentalities Successful programs emphasize gathering buy-in from stakeholders across departments into their drone programs -- and they do it by driving value. Almost every strong program we engage with has an “anchor story” about a specific event involving their drone program that drove immense value. Some examples are: a $50k reduction in insurance costs, a massive reduction in risk, and a deployment that would have cost $35k in helicopters. Documenting ROI and presenting it constantly when advocating for change is a key mentality embraced by expanding orgs. Once success has been achieved across multiple projects or areas of the organization with the drone program, program managers who are not afraid to ask for the resources they need are the ones that (surprise) get the tools to move forward. Technology investment requires risk, and to really expand, program managers need to take the success they’ve experienced and put it on the line to get the resources they need to really expand. One of the most compelling mentalities of successful programs is thinking “future forward.” Almost every team we’ve spoken to that has grown rapidly can articulate a vision of what their program looks like in five years, not just next month, or next year. Looking at the value that can be driven by drones on this more distant horizon, and considering, planning for, and advocating for the use of future technologies is a crucial difference between programs that expand slowly, and those that are gaining real momentum. Strategies So, as a program manager or stakeholder, what can you do to build your program successfully, and work around the issues we’ve broken down above? From our conversations with dozens of programs in the last few years, we’ve unpacked some key concepts that are worth ensuring you’re focused on. Process, process, process - Everything that your team does in the field and in the office needs a defined, documented process. We know it’s not easy, and that every switch out of hardware or software, or a new tech integration could change it, but repeating the same methods, the same way across your organization is the only path to true program scalability. Luckily, we’ve helped prepare some processes in later sections of this ebook to help. Carefully examining the technology - There are so many technical facets to drone data capture, from the hardware itself, to the sensors the drones carry, to post processing, data presentation, data analysis, hosting, and backups, etc. All of that is more than many teams (or IT departments for that matter) are comfortable with. Staying in the know and identifying key areas for you and the team to research and learn about monthly and quarterly is crucial. Research and evaluation need to be a continuing part of your program's growth. Enabling team members - To properly scale a team, delegating responsibilities is key. We’ve seen multiple programs find success by designating “leads” on their field teams to fully research and own the process for specific categories – like post-processing workflow, data storage, and equipment maintenance, etc. Focus on 100% mission success - The right mindset for knowing every mission will always capture data that makes it to the end user stakeholders is focusing on redundancy. This data comes from actual drone hardware and moves all the way across the workflow to data backup. The simpler your redundancy workflow (automatic backups, for instance) is, the more your team can focus on the other challenges in the process chain. Section 3 - The GNARBOX Solution GNARBOX Background and the problem we’re solving Originating from the Media & Entertainment industry, GNARBOX was first developed to replace the laptop for photographers and small production teams who need to back up their files in the field. In order to deliver a better solution than the laptop, it was essential that the product be more convenient to use, much lighter in both weight and bulk, and rugged enough to operate in harsh environments. Accordingly, by making a battery powered device that could fit in a pocket, operate in the most extreme environments, and with a single touch, back up multiple SD cards to multiple destinations, GNARBOX succeeded in reducing the risk of data loss significantly on shoots and in media productions. With the GNARBOX 2.0 Drone Edition, an evolved toolkit and workflow solution is being utilized by drone programs to not just protect against data loss, but expedite delivery times to the cloud for processing and project delivery. The following key constraints in an existing drone workflow are being addressed: Short Term Data Loss caused by problems in the inherent data structure MicroSD cards are not a reliable single source and are susceptible to loss, damage and corruption Laptops are not always a suitable backup solution in the field due to weight, durability and power requirements Cloud uploads, while a key element of the backup process, require continuous reliable low latency that is not available in all field operations Sorting & organization of content requires a lot of time to complete on a limited timeline and overtime or poor organization can lead to data loss. GNARBOX workflow breakdown The GNARBOX workflow is optimized for convenience in the field, but to get the most from this innovation, settings must be configured to a program’s liking at initial setup. Further, both field and office workflows may be structured for on premises or cloud backup and storage. Initial setup prior to field use: Power up and perform any required updates to the latest software Configure backup settings to include/exclude: Smart Backup (recommend ON) xxHash64 Checksum (recommend ON) MHL Logs (recommend ON) Flatten Folders (recommend ON) Folder by File Extension (user preference) (optional) Authenticate Pix4D or Dropbox Cloud Connection Configure Backup Presets and assign default preset Configure folder structure for organization (optional) Select cloud destination for automatic uploads Make sure to have a microSD to SD adapter in the GNARBOX SD port Field workflow (without cloud upload): Power up Insert MicroSD card Hit “Backup Now” when it appears on screen IF you are backing up to a second external SSD that is plugged into the USB-C port, go to Advanced Backup at this step and select “multi-destination copy” Eject and power down once the backup and checksum is completed Field workflow (with cloud upload): Power up Connect iPhone or iPad with ‘host spot’ enabled using a Lightning to USB-C cable Insert MicroSD card Hit “Backup Now” when it appears on screen IF you are backing up to second external SSD that is plugged into the USB-C port, go to Advanced Backup at this step and select “multi-destination copy” Eject and power down once the backup and checksum are completed Power down and disconnect phone once the cloud upload is completed Office workflow (backup to laptop/desktop): Power up Switch GNARBOX into Mass Storage Mode Connect to laptop or desktop using the right-side USB-C port Office workflow (for upload to cloud or NAS/Server): Power up Connect GNARBOX to USB-C to Ethernet dongle Connect the GNARBOX App to the GNARBOX’s WiFi signal Select and upload files/folders to Pix4D, Dropbox or configured NAS Servers Benefits Users are seeing from GNARBOX in the Field There are four categories of practical benefits for drone programs: Reduced risk of data loss: For UAV program managers and pilots operating in the field, the convenience, speed and reliability of the GNARBOX prevents you from having to rely only on microSD cards or bringing your laptop to secure data, check for corruption and review flights. As a result, operational costs are saved by not having to reshoot any location. Improved chain of custody: For Public Safety operations, the GNARBOX simplifies the chain of custody process and hardens it compared to handling all evidence on only microSD cards until it reaches command and control. Faster delivery times: Leveraging the connectivity and cloud upload capabilities of the GNARBOX, teams are able to begin file delivery from the point of capture rather than after traveling back to a hard line or WiFi connection for upload. Weight savings: Removing the laptop and hard drive combination from a kit generates immense weight and space savings particularly for teams that need to travel greater distances. What Types of Drone Teams Benefit Most from GNARBOX? GNARBOX has delivered benefits to teams across all industries including Utilities, Transportation, Public Safety, Construction and Energy. Some key characteristics that drive greater opportunity for these teams include the amount of travel, the size of the team, the environments they operate in and the amount of data captured. Travel time: The greater the average travel time per shoot, the more need there is for paring down one’s kit to essentials to make each trip as efficient and effective as possible. Operating environments: For teams that are frequently operating in more harsh environments in which laptops are no longer able to be used. Size of team: The larger the team, the larger the issue of standardizing digital asset management best practices becomes. GNARBOX can streamline file naming, folding and chain of custody processes for large teams. Size of data capture: For teams capturing large amounts of data, having an offload destination immediately on hand allows you clear valuable space on microSD cards between flights. For teams capturing small amounts of data, GNARBOX can be leveraged for immediate file delivery to the cloud. Specs 1TB NVMe SSD USB-C (x2), SD and Micro HDMI ports USB-C transfer 390MB/s & SD backup 75MB/s Water, Dust & Shock resistant (MIL-STD810) Single-step backup with status and checksum 802.11ac WiFi 5 for phone, tablet and cloud connectivity 2.4GHz Intel Processor, 4 Core Intel HD Graphics .DNG image / H.264, H.265, ProRes support Removable battery for 3-6 hours of use Integrates with Pix4D & Dropbox for cloud uploads If a particular drone program might benefit from features and functions like these, teams should consider several important digital asset management criteria and overall strategies described in the following sections. Section 4 - Strategies focus on Directories, Folder and File structure and Naming The emergence of digital image data preceded the consumer drone industry by nearly two decades, with initial systems for wrangling and managing media developed by lifelong broadcasters and cinema warehouse masters that crafted the art of structure for endless reels of video, audio, and still frames. Film Loaders and Media Managers transitioned into Data Wranglers and digital imaging technicians (DITs) as media transformed from tape into spinning hard drives and solid-state memory cards. While the content captured and retained grew in quality, the inclusion of metadata that could be annotated to a file and then extrapolated into useful information beyond what was seen visually - added immense value to each frame. This new data richness was unlike previous analog media in that the ability to view, approve or disapprove, and reshoot content could be exercised nearly instantaneously without time-consuming and costly processing. Additionally, this EXIF data would lead eventually to autonomously derived transforms that would reshape data analytics and mapping capabilities as the price of digital imaging quickly fell. Given that over a century of content was captured for TV broadcasting and the worldwide motion picture industry before the digital era began, many of the original systems for media management that were developed have been refined and remain in use today. Efficient Data Management and Structure Manual creation of folders and files for digital data should follow the best practices described earlier – not using automated assistance. This process is what is currently in place without GNARBOX or some form of automated media management. In the early days of Hangar, we would have to create a directory structure, such as the one described below, for our pilots to use, just so that they would not lose data, and to make sure that it was going to the right place. A lot of demand side platforms (DSPs) still operate like this, as does film & TV production, although some tools are coming online to automate directory setup. The outline below can serve as a tip or guide for someone who isn’t using some form of media management in directories. The breakdown below first illustrates and exemplifies the full tree - more of what a content manager or editor directory hierarchy setup should look like - Short-form Breakdown Owner (Stakeholder / Client Directory - the owner of the project) Project (Master Subdirectory - movie title, construction site location, etc.) Task (Subdirectory Focus - can be location, scene, or mission type) Asset (Equipment Subdirectory - which drone or camera used) Time / Date (subfolder for Take# or recurring captures) File Example for Drones in AEC HPI Client 303 Congress Daily Image Series Mavic013 2020-11-01 DJI_00167.jpg … … For operators and handlers alike, the above procedures can be either simple or incredibly tedious. This is determined by the mission type, how much of it is automated vs. manual, and what the daily volume of expected data is. There are dynamics at play that determine this complexity, which tend to break down as follows: Operator data dynamics How many pieces of capture equipment am I as an operator using at a site? How many capture sessions or missions am I flying in a day? How is my storage diversified? Am I using one or many SD cards? When do I upload or backup my data? End of a take? End of the day? Where does my data live or get backed up while we’re in the field? How do I pre-sort this data before uploading it? Is any of this process automated for me? Common Operator Process today Format media in prep for shoot Mission / Shotlist Review (manual or automated) Capture content (one or multiple devices) at location. Dump media between captures, change SD card, or continue with same SD card. Continue capture (next content or next mission location) Sort all data manually or a portion thereof. Upload data to drive, processing, or deliverable location. Process data (if operator also is handler) The data handler, whether they are the operator, coordinator at a DSP, or an on-set DIT, is always involved between steps 5 and 7 of the Operator Process above. Structuring and maintaining a data repository is a shared responsibility to ensure that loss does not occur due to misallocation or non-capture. As such, developing a file structure and naming convention set is vital to a successful and efficient data transfer process. This system, however, has multiple points of potential failure. In order to improve upon this situation, there are several key areas where efficiency can be gained by the operator or handler, and an assurance of data integrity can be added: (Op) Move data frequently - Don’t wait until the very end of the day to start the sorting process. Swap cards and create new folders between shoots. A pause not only keeps data easier to sort, but can also serve as a checkpoint to verify that scheduled content was captured. This reduces fatigue and error. (Op & Handler) Create Preset folders where content will live, based upon known deliverables from the shoot / mission requirement. This can be accomplished manually, but automating this repository creation by linkage to schedule, or directly to a mission planner or capture app is more efficient. (Op) Use tools such as the GNARBOX to automatically sort data by location captured at/on specific times and dates, and by specific assets - rather than just by chronological order of capture. This can greatly reduce man hours spent thumbing through content prior to upload. (Sole Op and/or Handler) Create naming conventions for files that highlight the included content rather than just the sequential order. Company internal procedures for this function that create content batch naming types for processing is a huge win. While the process is manual at first, later changes for ingest can add more functionality and automated controls. The holy grail of file structuring and data sorting is reached through the use of tags automatically generated at the capture event, or through an embedded post-handler. This functionality requires an imprint edit of the metadata in-app which gets written to the newly captured file in real time. Hangar and DroneDeploy have done similar work on this in the past to add a small identifier that assigns a value to each file - which traces the data against the defined trigger action, and databases the list of expected files against what is actually captured. Drone manufacturers and software developers need read/write access to captured files on the drone and/or controlling device to implement this (via SDK). Such an auto-routing enablement eases the burden on the operator to the point where the only remaining task post-capture is to upload without any sorting, but this process requires significant software development today. Upon upload, the virtual data handler sorts the files into their respective bins for analysis and processing. --- Section 5 - Strategies focus on Establishing Asset control, Checks and Validation Whether it be an enterprise entity whose sole focus is the delivery of actionable mapping data in construction, or a production company that is working on the next Hollywood blockbuster - asset control and the ability to validate the fact that that content was captured without compromising quality is vital to the success of an organization. Proper file structuring is only the gateway to how this is accomplished and made efficient. Deciding on how files are nested is further enabled by a system that recognizes and tracks the equipment used for data capture, analyzing the health of storage media, and examining the quality of the content as it is stored and transferred between data repositories. Every resource and piece of equipment used in this process, down to the data itself, is an asset that needs a system of control. While we explore automated checksums for data quality in a later section, it is important to understand that the movement of files from capture to processing and deliverable content is an interwoven system of moving pieces that cannot function well with any missing or faulty component. A given organization often requires the use of varying size and capability-driven drones, outfitted with a variety of sensors and lenses for the job at hand - in addition to supporting ground equipment. As if that level of complexity isn’t enough, many commercial ventures and organizations combine multiple crews at concurrent capture locations using a variety of similar or different pieces of equipment. While Asset Control is a function of File Structuring & Management, this process ensures that the movement of data takes place with the fewest number of steps, thus reducing the potential for lost, duplicate, or erroneous information. Useful tools can be incorporated into Asset Control that also can serve as preventative and predictive measures to ensure that the health of storage media, and even an organization’s fleet of drones, are maintained. By doing so, many efficiencies can be achieved, and data ultimately can be delivered at the highest quality possible. What’s at stake? An organization's assets not being tracked can lead to - Duplicate data uploads, even with a good file management system Use of incorrect format or specified settings (without precapture checks) Older (many read/write cycled) storage media prone to failure and data loss Use of incorrect or improper equipment - nullifying capture Loss of aircraft due to a lack of proper predictive or preventive maintenance Lost data… from equipment failure, lost media, etc. from failure to back up data due to not tracking storage space on a backup device (already mentioned, but important to reiterate in this section) Slippage or Shrinkage - loss of equipment from theft or incorrect use; loss = lost data capture capabilities (Same as above, with a slight variance addressed below) What is the Remedy? Routine manual or automated backup of data and card formatting is the #1 step to ensuring good file handling etiquette A capture plan or shot list that specifies capture settings should be used as part of any pre-flight or precapture checklist to ensure appropriate content is collected Use field tools and/or analytics software and asset control suites to check and ensure that storage media read/write cycles are known. Knowing and anticipating remaining lifespan for storage media is a preventative measure against data loss. Backing up data while on-capture is essential. Having tools like GNARBOX to automate this process greatly improves efficiency and serves to protect data, as well as strengthens the asset control chain. Pre and post capture inventory routines, assisted by a software-based management system, are an invaluable tool for not only the reduction of lost equipment, but also for facilitating maintenance scheduling. Several tools are available that, when used to their full capability, allow users to make timely decisions based upon predictive and preventive operations such as needed calibrations, motor and battery replacements, and stop-use for faulty storage media. Exemplified Tools GNARBOX Automated sorting & uploading of data, also a valuable check for duplicate data, card formatting Airdata Flight log tracking and powerful analytics tool for drones. This is the best method of maintaining aircraft with minimal cost, or additional manpower. Industry and regulatory friendly, as this tool works with many manufacturers of drones, not just DJI Roles, license holders, and personnel management Hard Disk Sentinel (HDSentinel) Software monitoring tool to detect errors in storage media EZOfficeInventory Awesome tool for tracking AV gear, survey equipment, drones, tripods, etc. Check-out, Check-in logs Preventative maintenance and analytics tools built in Section 6 - Strategies focus on Choosing the right Cloud Provider Manned edit bays and dedicated processing stations that handle the rendering of complex orthomosaics often require purpose-built workstations with immense storage space and top-of-the-line processing capabilities. Regardless of whether or not your business handles this internally or is offloading these tasks to cloud-based service providers such as DroneDeploy, the sheer volume of data that is generated by today’s imaging sensors has exponentially increased in parallel with gains in image resolution and capture equipment that an organization is using. This trend amounts to terabytes of content that is constantly in motion - being edited and transformed, duplicated, published, and eventually archived for later use. It’s no surprise that storing this data in any single repository comes with an inherent risk of loss due to hardware faults or power outages. Additionally, multiple users often need to access data at the same time, which can slow down even the most robust internal servers. With many employees of an organization now requiring this access to be predominantly remote, this added strain can only be solved by moving some, or all captured data into cloud-based repositories. Whether or not your organization has already embraced cloud storage, or you are looking to derive greater efficiencies through restructuring an existing repository, there are key benefits to making the transition, as well as helpful tips to ensure the move is as seamless as possible. Integrating cloud-based storage for your organization’s content can be painless and without much workflow disruption - so long as several steps are understood and carefully taken. Do I need Cloud Storage? How much data am I moving? - The industry and type of business greatly affect the answer, but insight often comes from the size of deliverables. Orthomosaics and video content are obviously storage vampires, but aren’t alone in that. If hard drives are a line-item in monthly accounting meetings, this is probably a sign that you need cloud storage. How many users need access to data at the same time? - An answer greater than one means “yes… you need cloud storage.” Are my employees working remotely, or sharing files via email? - If the answer is yes, you would really benefit from cloud storage. Am I constantly deleting files to make more room for new data? This is a red flag. Deleting data to make more room means you are rewriting data to a physical piece of hardware and risking data loss from compression or an eventual failure. Does it take me or my employees a long time to find needed data? While the short answer is to embrace a better file management and asset control system (See above), the long answer is yes. You need cloud storage. My organization maintains sensitive data, with strict security protocols. You definitely need cloud storage, as many providers have enterprise-level security and encryption products that increase, not decrease data security. Single-point failure systems can be compromised with greater ease than cloud-based solutions. What are the benefits? Cloud-based storage can handle enormous amounts of data quickly, and with no additional hardware. Many providers, depending on the tier of service selected, can accommodate as many users accessing the data at the same time, as necessary. Since cloud storage is not localized, your data lives in redundant locations that allow for lightning-fast transfer times. Data stored in the cloud is accessible anywhere on Earth, with the only requirement being an internet connection. Many providers offer small software applications that allow for offline-synchronization that can be very useful for those on-the-go, pausing and resuming transfers when internet access is available - but keeping local, secure copies of data in temporary storage on your devices. As your organization’s data grows, so can your storage plan. All cloud providers have options for overage and plan enhancements as you need them. This works both ways, too. If you need less storage, you can downgrade your capacity requirement and scale as you need to. Cloud-based storage providers embrace good file management practices. Your workflow is yours, and access is easily controlled by a dedicated administrator. Data security is not only a capability of all cloud-based storage providers, but a competitive driver for their sustainability. In order to bring in customers continually, these providers are self-motivated to continually improve their offerings. They proudly boast about their security accolades and organizations served. How do I get ready to make the move? Embrace File Structuring - If your organization has not developed a system to organize data in an efficient manner, this is a first step. The move is only as painless as the process by which you manage your existing data. (See topic above for the in-depth) Set Goals - Identify the reasons a transition is important for your organization’s success, and what key benefits you want to achieve over time. Weigh the pros and cons - Schedule demos, ask hard questions, and decide on a provider Inform your organization - Communicate when, how, and why you are making the move. Educate your team on new processes, accept feedback, and give users time to learn about the chosen platform before you implement it. Schedule a Move Date & Transition Period - Prepare a task list, appoint an administrator, and make a plan to move older, or less consequential data first. The process should be gradual until it cannot be - with a final move to cloud storage to take place during a low-impact or offline period. Keep the data you already have stored locally - It goes without saying that redundancy in data storage is the most important imperative in this entire document. Keep a local copy of your data throughout the transition, and then archive it. Section 7 - Strategies focus on Data Preservation Through Checksums The flow of data occurs from capture to processing, rendering to repository, and inevitable migrations between hard drives and cloud storage centers. This repeated transmission of vital files inherently comes with some expected data loss due to a variety of causes. While there is no perfect solution that offers true lossless transfer, the preservation of data integrity hinges on the tools used early-on to mitigate and minimize this risk of loss. What are the most common ways loss occurs, and how can an organization use an understanding of the preceding topics we have explored to ensure that its data is protected while in motion? What tools exist that can provide assurance of critical data preservation? Let’s first address this by answering what a Checksum actually is. Demystifying Checksums Since digital data cannot have an expectation of 100% preservation after any transfer, a variety of tactics and protocols have been created that check for errors during or after a movement of data occurs, and they differ in operation depending on the type of connection used. A checksum is a small block of data pulled from another block of digital data that is used to detect any missing or erroneous information that may have been introduced during its transmission or storage to a new location. In this way, a checksum is a measurement of data preservation, and a confirmation of data file integrity. By using a checksum utility, a baseline measurement is established and allows for data monitoring to ensure proper retention. When Are Checksums Useful? Checksum utilities are available to diagnose these errors that occur during transmission or storage. Preservation of data is something that should be considered from the very moment you move captured content to a storage drive. When checksums are applied early, this value can move with your data as a snippet to validate its accuracy over time and migrations. There are a variety of ways that data preservation is susceptible to loss or error over time. For example, a file might not have properly downloaded due to network issues, or hard drive problems could have caused corruption in a file on disk. Different types of wireless transfer protocols such as UDP and TCP place differing levels of emphasis on latency vs. quality and can lead to unintended loss due to a misunderstanding of how data is transferred. If you know the checksum of the original file, you can run a checksum utility. If the resulting checksum matches, you know the file you have is identical. [Call-out to GNARBOX Checksum verification features?] Checksum Utilities in Practice Captured content should first go through a checksum utility to generate a data fingerprint that can be stored alongside data in a dedicated folder. Use the checksum utility to check data periodically or routinely after a data migration occurs between hard drives, or cloud service providers. Keep redundant copies of data with validated checksums in the event of data loss during transfers. Checksum Applications Fixity - Subscription-based utility, perfect for organizations. Automated features built-in FastSum - tier level, simple application ExactFile - Freeware for PC, allows for file checksums individually, or in groups Conclusion The road ahead for drone team leaders is an exciting one. As adoption increases, technology improves, and programs grow, there will be no shortage of challenges, but each of them can be met in the name of acquiring better data to drive strategic decisions and increasing safety by keeping teams on the ground. Data workflows may not be the most glamorous part of the job, but they are crucial to building process-driven, scalable drone programs. We hope this ebook has provided you with both ideas on how to better grow your program and tools that will enable you to avoid losing data in the field while managing your data more effectively overall. Questions? Ideas? Comments? Don’t hesitate to reach out to us. gnarbox.com Guinnpartners.com