In a field where public good and private profit exist side-by-side—where national security and business interests have the potential to contribute to/conflict with civilian interests—the ethics surrounding satellite imagery and remote sensing have been remarkably difficult to disentangle. With the launch of the National Aeronautics and Space Administration’s (NASA) first US satellites, what was initially aimed for federal and scientific research companies such as the United States Geological Survey (USGS) has grown into a multibillion-dollar private sector industry servicing state and local agencies, corporations (like oil and mineral companies), and private firms by offering processing and analytical services of raw data.¹ The benefits of Earth-imaging are clear: in addition to its applications for defense and intelligence, it yields important humanitarian benefits and improves life on Earth with the monitoring of disaster relief efforts, refugee movements, environmental degradation, and international disputes. However, the increased application towards profit-making ventures has shifted Earth observation (EO) from global protection to domestic spying, where nations such as the United States maintain financial incentives to use satellites to monitor citizens and corporations in order to track consumers and competitors. Geospatial technology using Earth-imaging satellites have imaging capabilities approaching those of military spy satellites, but commercial satellites possess one key difference: their operators are willing to sell the images to anyone, and there is a lot of money to be made this way. (Fig. 1)

The planet’s surface is a rich pool of data waiting to be mined. Cash-strapped municipalities and trend-forecasting businesses ranging from retail to agriculture have discovered the value of this imagery for tracking economic activity and the flow of human capital. As more and more satellites set their sights on the globe—taking millions of indiscriminate images—the issue of privacy comes into sharper focus. According to joint data gathered by Pixalytics in 2018, over 40 percent of the 1,980 active satellites currently orbiting Earth are listed for commercial use.² The same data shows that in just one year, there has been a 10 percent increase in the number of satellites launched for the sole purpose of obtaining planetary imagery.

Legal Gray Area

Like all data, satellite imagery is neutral until applied. The argument for the legality of satellite imagery is that it cooperates with all on equal terms; it does not change its story based on the inquirer. The fact that satellites continually orbit the globe, collecting data indiscriminately, implies that any image taken and stored in a routine sweep might not necessarily be categorized as directed surveillance. But if an interested party such as a law enforcement agency or Wall Street hedge fund were to specifically task a satellite provider to perform targeted surveillance, or a satellite was purposefully moved to acquire a specific image, then the case must be reconsidered.

This issue brings up the question of intent—but whether it is in the image collection phase or data processing phase, a line needs to be drawn with respect to what constitutes an invasion of privacy. In Privacy and Freedom (1967), Alan F. Westin defined privacy as “the claim of individuals, groups, or institutions to determine for themselves when, how, and to what extent information about them is communicated to others.”³ Our personal information is not a commodity to be bought and sold—at the very least, not without our knowledge and consent.

The Race to Rule the Skies: Politics of Outer Space

To understand the standards under which satellites and imagery distribution operate today, it is important to look back at the history of the industry’s global regulations, beginning with two policies set in place by the United Nations. The first was the 1967 Outer Space Treaty,⁴ which declared that outer space cannot be claimed as national territory and legitimized satellite travel by any country over any point on Earth. The second, a principle ruled by the United Nations General Assembly at its 95th plenary meeting in 1986,⁵ denied the right of prior consent for nations to review and possibly withhold data about their territories. These policies meant that, under international law, countries have no grounds for objecting to being imaged from space but have the right to access those images “on a non-discriminatory basis and on reasonable cost terms”⁶ or in other words, for a price.

So began the international race to rule the business of remote sensing technologies. In the United States, the value of satellite imagery was pulled between conflicting interests of commodification and self-defense. When should satellite imagery be treated as a public good under the government’s provision, and when should it be treated as a private good to be sold for financial gain?  Economic interests want to maintain dominance in a large and highly profitable industry that the United States has pioneered. National security interests want to prevent potential adversaries from using the imagery against the United States or its allies.

Initially, policymakers tried to do both. The Land Remote Sensing Policy Act of 1992 privatized the industry at home, encouraging domestic companies to build and launch commercial satellites with higher technological capabilities.⁷ However, Cold War fears lingered, and a provision was written to allow the federal government to exercise “shutter control.” In the case that the government believed national security to be compromised, licensees would then be obligated to shut their satellite’s eye and limit data distribution.

America’s tightening grip on export control only drove business to foreign companies who didn’t fall under the same US regulations. The government was forced to reevaluate the economic constraints put on its home industry, especially since the military relied heavily on private sources for satellite intelligence, and therefore had a stake in their commercial success. Concerned that foreign competitors such as France, Russia, and India might dominate the market, President Clinton loosened restrictions on the sale of high-resolution imagery in 1994.⁸  Since then, policies have continued to favor release limits on commercial resolution in order to preserve US dominance in the market. The government’s inclination toward commercial interests over its own national sovereignty may shed light on how it overlooks the privacy of its citizens.

Earth-Imaging Unlocks the Earth-Observation Frontier

Commercial privatization completely disrupted the Earth-imaging industry by injecting healthy competition and venture capital into what was previously a very conservative, state-run environment. Costs plummeted in all aspects of space technology, from capturing devices to launch services. Ten years ago, it cost $200 million to build and launch a satellite into orbit. Today, it costs around $200,000—a small investment in a global space economy worth $383.5 billion.⁹

In 2000, the US Department of Commerce gave geospatial-imaging juggernaut DigitalGlobe permission to sell pictures (Fig. 2) at 50-centimeter (19.6 inches) resolution, and in April 2014, they were cleared to sell them at 31 centimeters (12 inches)¹⁰—non-US customers included. The move would unlock $400 million in additional market opportunities for the company and allow purchasers to go from being able to identify a car to being able to identify its model. When our lawmakers surrender control of domestic territories to remote sensing, are they also forfeiting an individual’s right to privacy?

The success of the Earth-imaging industry spawned an eager Earth-observation industry whose main concerns are processing the former’s data. Startups like Orbital Insight, Descartes Labs, Cape Analytics, and SpaceKnow provide value-added services to satellite imagery through “EO + ML,” or AI-driven data analytics for earth-observation using machine learning and computer vision techniques.¹¹ The technology eased the heavy expense of imagery analytics and interpretation previously performed by humans. But the combination of all-seeing satellite imagery with sophisticated computerized number-crunching comes with a significant potential for abuse, especially when motivated by profit. Regulatory structures have yet to meet the industry’s explosive growth, and this uncharted territory may be giving leeway for analytics firms to maneuver lawlessly through the data.

Perhaps It’s Personal: Three Use Cases

Crop Yields and Commodity Trading

By pairing satellite imagery with a method called imaging spectroscopy, which analyzes material absorption and reflection rates from Earth’s surface, a wide range of data can be extracted from agricultural areas.¹² Colorful image analyses reveal everything from vegetation and soil health to water abundance—and even specific plant species—at a 97-percent accuracy. (Fig. 3) Imaging spectroscopy, however, is not limited to vegetation. The tool allows any material or substrate to be mapped, whether natural or manmade, as long as it displays unique absorption features.

Often hidden behind the guise of “smart farming” tailored toward empowering farmers, satellite-enabled crop monitoring has a customer base composed largely of commodity traders, insurers, and market-research firms. With agriculture-related intelligence, a farm-management and forecasting system can be assembled by any businessman, even without a previous track record in the industry. Wielding a library of satellite imagery (some dating as far back as 30 years), companies can check the performance of a particular field over several decades.¹³ These records of productivity—including soil-data, weather forecasts, and the known behavior of individual crops strains—are cross-referenced with the field’s biomass during the current season in order to predict what the anticipated yield will be.

TellusLabs, a Boston startup recently acquired by Indigo Agriculture, is one such company aimed at streamlining agriculture data into the financial marketplace. Pitched as “a Bloomberg terminal for Earth signals,”¹⁴ it uses satellite imagery and machine-learning algorithms to generate intelligence about natural resources. The startup’s main product is an interactive, online dashboard that shows key financial indicators, such as predicted yield, harvested area, and total production. (Fig. 4) Paying customers can easily navigate through the data by state, agricultural district, or county level.¹⁵ Since satellite imagery covers entire regions and even countries, and is often updated at least once a week, these tools can forecast the size of the world’s crop output before it is harvested. Satellite imagery affords powerful financial and political information capable of controlling market price models strategic in determining areas of investment and effective in targeting products and services to farmers.

Consumer Shopping Habits and Retail Forecasting

 

In another technique called “parking lot fill rate analysis,” remote-sensing metrics companies combine artificial intelligence with the satellite imagery of parking lots to predict revenue for hedge funds and Wall Street analysts (Fig. 5). By pinpointing the exact location of a store’s entrance and counting the cars month in and month out, analysts are able to get a fix on a company’s customer flow and shopping habits. From there, they can apply mathematical regression to forecast the company’s expected sales before official statistics are released.

Orbital Insight, currently the most prominent geospatial analytics company, officially monitors the parking lots of more than 200,000 individual store locations for more than 100 different chains across all industries, including food, automotive, and retail.¹⁶ The lots subjected to the company’s so-called “state-of-the-art car detector” include The Cheesecake Factory, Chuck E. Cheese, Gap, Office Depot, O’Reilly Auto Parts, and The Container Store (Fig. 6). A patron’s decision to shop either at Walmart or Target may help an investment firm decide where to put its financing. While investments in your local retailer may seem only moderately significant, it is important to note that vehicle statistics can be a powerful proxy for tracking urban development, population, income, and gasoline demand.

Governments Surveilling Private Property

Satellite imagery has also become a tool for cash-hungry governments. Faced with budget crunches, governments in the United States and Greece¹⁷ have used Google Earth to search residents’ private properties for undocumented assets such as swimming pools, giving officials an excuse to collect extra taxes. Such was the case in the town of Riverhead, Long Island, New York, where authorities utilized Google’s publicly available imagery to search backyards and found 250 pools had been built without permits, collecting $75,000 in fines.¹⁸ (Fig. 7)

In the city of Adelaide, Australia, a similar but unusual case took place. Residents’ properties were scoured from above for pools, but this time by a team of machine learning algorithms and human volunteers from around the world. The campaign was led by Tomnod, a subset of DigitalGlobe that crowdsources volunteers to identify objects and places in satellite images, and is sponsored by a private company that compiles public and private sector data for a variety of markets including education, public safety, government, telecommunications, and insurance.¹⁹

By training an algorithm on sample “chips,” or image cutouts of individual properties, the company masterminded an application called PoolNet to classify which images contain a pool.²⁰ (Fig. 8) PoolNet has the capacity to accurately distinguish pools that are empty, those whose water have turned green, or even those covered by tarps. This model, capable of identifying 60 properties per second, would provide an efficient and reliable framework capable of gathering much more information than just pool counts. The presence and actual inventory of vehicles, boats, and solar panels were listed as valuable items to track in a future phase.

Conclusion

The Fourth Amendment of the US Constitution maintains that “the right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated” and affirms the need for “probable cause” in the event that government officials want to search your property.²¹ Likewise, any unlawful entry onto private property would be considered criminal trespassing. But somehow the silent, faceless monitoring from above has been plainly disregarded and excused by overstepping these constitutional rights. Cases like these may be setting a dangerous precedent for future surveillance strategies. If these instances have been judged permissible by law, what is stopping other companies and government agencies from monitoring their citizens?

As with most Big Data collection and use, regulation needs to keep up with the pace of technology and recognize the collateral impact—down to the individual—that it has on privacy and personal freedoms. The matter is a moving target: there is currently a bill up for passage in Congress that could potentially cede more power to the satellite industry. The bill, titled the American Space Commerce Free Enterprise Act of 2017, would provide a faster licensing timeline and would put the burden on government to prove why a company should not get a license rather than require a company to prove why it should.²² It will be a challenge to balance the opportunities of Earth-observing technologies against the rights of citizens, but safeguards must be put in place before something unimaginable happens, and this behavior is normalized.

Endnotes

¹ James A. Vedda, “U.S. National Security and Economic Interests in Remote Sensing: The Evolution of Civil and Commercial Policy,” The Aerospace Corporation, February 2, 2009, https://fas.org/irp/eprint/remote.pdf.↵

² “How Many Satellites Are Orbiting the Earth in 2018?” Pixalytics, August 22, 2018, “http://www.pixalytics.com/sats-orbiting-the-earth-2018/”>www.pixalytics.com/sats-orbiting-the-earth-2018/.↵

³ Alan F. Westin, Privacy and Freedom, (New York: Atheneum, 1967), 7.    ^↵

⁴ United Nations, General Assembly, Treaty on Principles Governing the Activities of States in the Exploration and Use of Outer Space, including the Moon and Other Celestial Bodies, RES 2222 (XXI), 1499th plenary meeting, December 19, 1966, http://www.unoosa.org/oosa/en/ourwork/spacelaw/treaties/outerspacetreaty.html.↵

⁵ United Nations, General Assembly, The Principles Relating to Remote Sensing of the Earth from Outer Space, Principle XII, A/RES/41/65, 95th plenary meeting, December 3, 1986, “http://www.un.org/documents/ga/res/41/a41r065.htm”>http://www.un.org/documents/ga/res/41/a41r065.htm. ↵

⁶ United States Congress, House Committee on Science, Space and Technology, Land Remote Sensing Policy Act of 1992, H.R.6133, 102nd Cong, October 28, 1992, https://www.congress.gov/bill/102nd-congress/house-bill/6133.↵

⁷ The White House, “US Policy on Foreign Access to Remote Sensing Space Capabilities,” Presidential Decision Directive/NSC-23, Federation of American Scientists Intelligence Resource Program, March 9, 1994, https://fas.org/irp/offdocs/pdd/pdd-23.pdf.↵

⁸ Valery Komissarov, “AI Applications for Satellite Imagery and Satellite Data,” Tech Emergence, September 10, 2017, “https://www.techemergence.com/ai-applications-for-satellite-imagery-and-data/”>https://www.techemergence.com/ai-applications-for-satellite-imagery-and-data/.↵

⁹ “Space Foundation Report Reveals Global Space Economy at $383.5 Billion in 2017,” Space Foundation, July 19, 2018, https://www.spacefoundation.org/news/space-foundation-report-reveals-global-space-economy-3835-billion-2017.↵

¹⁰ Andrea Shalal, “DigitalGlobe gains U.S. govt license to sell sharper satellite imagery,” Reuters, June 11, 2014, https://www.reuters.com/article/digitalglobe-imagery/digitalglobe-gains-u-s-govt-license-to-sell-sharper-satellite-imagery-idUSL2N0OR2UX20140611.↵

¹¹ Komissarov, “AI Applications for Satellite Imagery and Satellite Data.”↵

¹² “The Future of Agriculture,” The Economist Technology Quarterly, June 9, 2016, https://www.economist.com/technology-quarterly/2016-06-09/factory-fresh.↵

¹³ Roger N. Clark, et al., “Imaging spectroscopy: Earth and planetary remote sensing with the USGS Tetracorder and expert systems,” American Geophysical Union, December 6, 2003, https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2002JE001847.↵

¹⁴ Elizabeth Woyke, “Crystal Ball for Corn Crop Yields Will Revolutionize Commodity Trading,” MIT Technology Review. August 9, 2016, https://www.technologyreview.com/s/602096/crystal-ball-for-corn-crop-yields-will-revolutionize-commodity-trading/.↵

¹⁵ Ibid.↵

¹⁶ “Orbital Insight Expands U.S. Retail Traffic Product to More Than 100 Retailers,” Orbital Insight, March 20, 2017, http://www.marketwired.com/press-release/orbital-insight-expands-us-retail-traffic-product–more-than-100-retailers-2203991.htm.↵

¹⁷ Daniel Steinvorth, “Greek Government Hauls in Billions in Back Taxes,” Spiegel Online, August 7, 2010, http://www.spiegel.de/international/europe/finding-swimming-pools-with-google-earth-greek-government-hauls-in-billions-in-back-taxes-a-709703.html.↵

¹⁸ Aaron Rutkoff, “Google Earth Used to Bust Illegal Pools on Long Island,” The Wall Street Journal, August 3, 2010, https://blogs.wsj.com/metropolis/2010/08/03/google-earth-used-to-bust-illegal-pools-on-long-island/.↵

¹⁹   Kostas Stamatiou, “Using Crowdsourcing and Machine Learning to locate swimming pools in Australia,” Tomnod, April 25, 2016, https://blog.tomnod.com/crowd-and-machine-combo.↵

²⁰   Digital Globe, “Pool Detection Using Deep Learning,” GitHub repository, November 30, 2016, https://github.com/DigitalGlobe/mltools/tree/master/examples/polygon_classify_cnn.↵

²¹ “Fourth Amendment,” Legal Information Institute, Cornell Law School, “https://www.law.cornell.edu/constitution/fourth_amendment”>https://www.law.cornell.edu/constitution/fourth_amendment.↵

²² United States Congress, House Committee on Science, Space and Technology and Senate Committee on Commerce, Science, and Transportation, American Space Commerce Free Enterprise Act, H.R.2809, 115th Cong, 2017-2018, https://www.congress.gov/bill/115th-congress/house-bill/2. ↵

---

Author Affiliations

Jacqueline Wu is a graduate student in Parsons School of Design’s MFA Design and Technology Program.