Tag Archives: gis

Processing Sentinel 2 satellite imagery in Norway

ANB-11-05_frontpage_shadowThe second report on “Preparations for acquisition and application of optical satellite data for Norway Digital” (Gjertsen et al) written for the Norwegian Space Agency has now been published.

This report is a continuation of the work presented in the report “Preparations for acquisition and application of optical satellite data for Norway Digital” (Trollvik et al., 2012). The main goal has been to specify the requirements for a national satellite data centre for optical satellite data from the Sentinel-2 and Landsat series Earth observation satellites. The main objective of a national satellite data centre is to facilitate easy access to and use of Sentinel-2 and Landsat 8 data for Norwegian users. Continue reading

Clearinghouse for the environment – the girders (II)

In February 2012 I wrote about “Environmental Spatial Data Infrastructure” on this blog. Later that year the case complex matured somewhat and in August I wrote the posting “Clearinghouse for the environment – the scaffolding (I)“.

Since then I have together with my colleagues had the opportunity to test systems in full scale by contributing to the implementation of clearinghouses in partner countries.

Last time I showed how a stack consisting a hardware layer with vmWare as one of the basic modules could form the basis of an environmental spatial data infrastructure. In some ways this was a rather optimistic setup.

A stack of hardware and software which could become very usefull...

Some of our main challenges with the above set up was maintenance of physical equipment. So we removed that layer. Maintaining a complex setup with a virtual machine environment, or getting access to local environments proved in general to be difficult. So we ditched it.

To get the systems running we needed:

  • Shared access to the systems for administrative purposes
  • A flexible backup-system
  • An option to duplicate successful setups
  • Scalability
  • High availability
  • Flexible security system

Other things we considered important

  • The system should not tie our partners up in future licensing costs
  • Compliance to central standards
  • An option for partners to move the systems to physical infrastructure if necessary
  • Option to keep traffick outside our own company networks – since they are de-facto external systems paid for by external partners

As you all can see in all a lot of considerations which we had to relate to.

AWS_LOGO_CMYK-588x214Since internet access across borders in any case would be relevant for retrieving external map layers we started looking at how we could use Amazon services. I already had experience in running virtual macines using Amazon EC2. Amazon helped us out with many of the issues mentioned above. So in short we moved the whole setup to Amazon. The following figure illustrates the setup.

sdi_onestop_amazon

In addition to the components relying on EC2 we have also found that using Amazon Simple Storage (S3) for storing survey data of some size could be a good ide. S3 allows the user to distribute files using “secure” links and even using the bit-torrent protocol for files up to 5 Gb.

We now have one such system built up and under testing. It looks good but as always the technology is but a small part of the equation. Establishing information flows, using standards etc represents the major parts of a national envoronmental spatial data infrastructure.

Should the need arise to develop custom made solutions it should be possible to add more virtual machines in the setup.

sdi_onestop_amazon

Given that our partners find this setup trustworthy we will probably suggest this as an entry level spatial data infrastructure for environmental data.

WordPress directly, and through countless plugins, supports many standards for embedding information. How information should flow between the different systems in this setup has been given some thought. I will try to elaborate on this in a later posting – hopefully in less than two years time.

QDGC shapefiles available for national and continental scale distribution

Aqdgc_logo new set of the Quarter Degree Grid Cell shapefiles has been generated. The update is global and delivers an error fix for the country level files as well as a new product – continent level files.

The QDGC shapefiles contain center lon/lat coordinates and the QDGC string for the different squares. The files are offered down to level four. For a country around the equator level four covers around 45 square kilometers with length and height a little under seven kilometres.

Read more about the use of QDGC on this page:

The calculations/export  this time took around 60 hours computer pricessing time including generation of world fishnet with the different sizes, square area calculations, assigning QDGC strings, compression and more. Continue reading

Rekreasjonsorientert bruk av droner – 2

Quadcopter etter en litt hard landing.

Arm på quadkopter etter en litt for hard landing

OBS: Artikkelen nedenfor er nå et par år gammel. Nye lover og forskrifte han ha blitt vedtatt. Teknologien beveger seg også videre.I en tidligere posting beskrev jeg om de juridiske rammene som omfatter privat bruk av et quadkopter (drone). I denne postingen vil jeg se litt på erfaringer gjort etter om lag 20 flyvninger med et quadcopter på en avsidesliggende gård på Mindland – en øy på Helgelandskysten.

Jeg vil belyse to av de viktigste forholdene rundt utprøvingen av quadcopteret. Det er riktig å starte med sikkerhet. Deretter vil jeg se på tekniske forhold rundt denne utprøvingen.
Continue reading

Rekreasjonsorientert bruk av droner – 1

Quadcopter uten monterte rotorer

Quadcopter uten monterte rotorer

OBS: Artikkelen nedenfor er nå et par år gammel. Nye lover og forskrifte han ha blitt vedtatt. Teknologien beveger seg også videre.

Denne artikkelen er den første av to knyttet til notater fra testing av en droner somrene 2013 og 2014. Min intensjon har vært foreta en vurdering av de mulighetene som et drone kan gi i hendene på en geograf som er interessert i naturforvaltning. Dette er gjort med privat utstyr – i fritiden.

Min erfaring var begrenset til flyvning av modellhelikoptre beregnet for innebruk. Men skal man lære må man prøve ut ting i praksis. Det kan være både morsomt og… litt skremmende.

Denne første postingen handler om de juridiske rammene for min aktivitet som på dette stadiet.

Artikkelen har senere blitt oppdatert flere ganger basert på endringer av rammene for lovlig rekreasjonsbasert flyvning med droner og ny informasjon, senest i september 2014.
Continue reading

Lokalkart for Mindland (heimstadkartlegging)

Skjærseth sett fra oven

Skjærseth sett fra oven

Det ble ikke heimstaddiktning denne sommeren. Ikke er jeg spesielt interessert i å skrive lokalhistorie heller. Men stedsnavn og kart har interessert meg en god del. Så da ble det heimstadkartlegging og ikke heimstaddiktning.

Øya Mindland er i likhet med de fleste andre steder i Norge godt kartlagt av Kartverket i samarbeid med kommuner, fylker og andre offentlige etater. Det er stort sett ikke mulig å konkurrere med kvaliteten på kartene fra Kartverket. Men dekningen av lokale stedsnavn er ikke alltid like god. Continue reading

American mink – alien species proliferation analysis

Procedure overview for the analysis

Procedure overview for the analysis

The aim of this posting is to document the more technical aspects of establishing the knowledge basis necessary to follow up the action plan against american mink (nevison vison) – an alien species in the Norwegian fauna. It will show how the Python programming language and relevant programming libraries (ArcPy and others) are used in an analysis aiming to understand where the mink can spread under given circumstances.

The motivation for this is to document the process for other relevant projects as well as to make relevant code and methodological descriptions available for other persons/institutions involved in similar projects. The work has been made possible with access to other freely available information online and as such this posting should be considered a timely way of paying back for “services provided”. Continue reading

Solving the ESRI arcpy dissolve challenge

ready_for_dissolveIn a recent project at work I did an analysis on the spread of an alien species in Norway using ESRI ArcGIS 10.1 SP1. In this particular analysis we assumed that the species could swim a certain number of meters in open sea. How would it spread and to what extent would current protected areas be invaded by this overseas stranger to our environment? The density of islands Norwegian archipelago is massive, so the possibility for the alien species to spread is rather overwhelming.

As part of the analysis I ended up doing buffers around islands in the Norwegian archipelago. After which it would be necessary to merge and dissolve the objects. This turned out to be problematic. But for some of the shapefiles I was working with ArcGIS (arcpy and python) simply failed to complete the dissolve operation.

After contacting our local ESRI representative, Geodata AS in Norway, they concluded that this was related to the following error in ArcGIS 10.1: NIM079373: Running a large number of features through the Dissolve or Buffer with dissolve option, hangs during process. I have not found any publicly information with this reference.

One could say that 7283 polygons is a tall order. One could perhaps also say that working with polygons in a task like this rather than with raster is asking for problems. Given enough time I will look into it – later – in that quiet week when nothing else is going on at work, sometime.

This blog post is about but  how I came to understand more about the limitations and possibilities with the ESRI arcpy Dissolve_management tool. It is also explains how I found a rather surprising way to make it faster.

Continue reading