Re: ArcGIS Limitation >> Re: [OKGIS] 10 Meter DEM's are available at USGS for most of the state.


Chronological Thread 
  • From: Peter Earls <earlsgis@yahoo.com>
  • To: Oklahoma GIS Community <okgis@gis.gis.ou.edu>
  • Subject: Re: ArcGIS Limitation >> Re: [OKGIS] 10 Meter DEM's are available at USGS for most of the state.
  • Date: Wed, 20 Oct 2004 16:31:09 -0500
  • List-archive: <http://gis.ou.edu/pipermail/okgis>
  • List-id: Oklahoma GIS Community <okgis.gis.ou.edu>

David, thanks for the original message a while back about the 10m DEMs being out there. I didn't know they were being created - it looks like the USGS is doing this for the entire country.

If anyone out there finds out that more of the state, especially Osage County, has been completed in the future, please send an update to this list. There's a data updates announcements page ( http://gisdata.usgs.net/ - once the data/map viewer is visible, click Help->Contents->FAQs and see first question to see this URL), but I haven't figured out how to get the update notification feature for bookmarked pages to work in my browser (Mozilla; if any one has had success with this feature, I'd appreciate hearing from you). So the only way I know of finding out when more data is out there is by trying the map viewer or checking the announcements page periodically. There's a question farther down the above FAQs page that says "You can setup a standing request within Earth Explorer, http://earthexplorer.usgs.gov to receive notifications of updates", but this seems to pertains to the 30m NED data only.

- Pete

==================================================================
Peter Earls
Department of Botany
Oklahoma State University


David Wheelock wrote:
I have also now discovered a limitation in the data size that can be handled by ArcGIS. I downloaded and mosaiced probably about 1/8 of the state. When I ran the Fill, Flow Direction and Flow Accumulation functions on the data in preparation for delineating watersheds I got some strange results which I reported to ESRI tech support. They analyzed the problem and determined that it's a bug related to the large size of the dataset. ArcGIS just can't handle that much data.
Currently, the only work-around is to split the data into smaller areas.
See the attachment for more details.

David Wheelock Application Specialist david@oda.state.ok.us <mailto:david@oda.state.ok.us>
Oklahoma Department of Agriculture, Food and Forestry
2800 North Lincoln Boulevard
Oklahoma City, OK 73105-4298
Phone: (405) 522-5477 FAX: (405) 522-3590
http://www.oda.state.ok.us/
----- Original Message -----
*From:* David Wheelock <mailto:david@oda.state.ok.us>
*To:* OKGIS List <mailto:okgis@gis.gis.ou.edu>
*Sent:* Friday, September 24, 2004 3:36 PM
*Subject:* [OKGIS] 10 Meter DEM's are available at USGS for most of the state.

I have discovered that the 10 meter DEM's are available for most of the state of Oklahoma. It looks like about 70-80% of the state. The people I've told about this have been surprised and didn't know, so I thought I spread the word through the list.
Here's where you can go to get it:

1. Click here: http://seamless.usgs.gov
2. Then click on the link that says: View and Order Data Sets -
United States Viewer 3. Then you'll see a map of the United states.
4. In the column to the right of the map, select the check box
for *1/3" NED*. That's the 10 meter DEM. Deselect the stuff
that's auto-selected, if you don't want it, anything else that you
do want.
5. Zoom in on Oklahoma where you want to get the data.
6. In the column to the left of the map, under *Download* select the
tool on the left,* Select by Rectangle*.
7. Select an area that you want and you'll either get a download form
or a message that you'll have to order it on CD's. For one county
you will have to download 4 chunks or so. And if you're area is
too large you can always break it up into separate data requests.

Regards,

David Wheelock Application Specialist david@oda.state.ok.us <mailto:david@oda.state.ok.us>
Oklahoma Department of Agriculture, Food and Forestry
2800 North Lincoln Boulevard
Oklahoma City, OK 73105-4298
Phone: (405) 522-5477 FAX: (405) 522-3590
http://www.oda.state.ok.us/

------------------------------------------------------------------------


*****
NOTE: An attachment named David Wheelock.vcf was removed from this message and deleted because it contained a windows executable or other potentially dangerous file type. Contact the system administrator if you need more information.------=_NextPart_000_00A2_01C4A24C.521A8360--

------------------------------------------------------------------------

Subject:
Re: #388730 (Matt F) winxp ai9.0 spat trying to run a field process in hydrology modeling tools in spat ana.
From:
"ESRI Support" <nimbus@esri.com>
Date:
Tue, 19 Oct 2004 12:11:33 -0700
To:
"David Wheelock" <david@oda.state.ok.us>

To:
"David Wheelock" <david@oda.state.ok.us>


*** Do not write below this line ***
David,

Our developers for Spatial Analyst have sent some info that point
this problem out to be a limitation of the data size. At ArcGIS 9.0 we are
now able to create an ESRI Grid larger than 2.14 GB. It does this by
internally blocking the pixels into tiles. But because of performance
considerations a VAT is not built for these Grids. With the size of the grid
you have it is probably creating the blocks but not building the VAT. This
VAT is a requirement for FILL, SINK and FLOWDIRECTION. So this is considered
a bug.

CQ00252902 - Sink, Fill, FlowDirection produce incorrect results for large
datasets

Unfortunately the only workaround is to physically split the raster
into smaller, overlapping pieces and run the analysis. I've gone ahead and
added you to the bug. Hopefully our programmers will figure a way around this
problem.

Regards,

Matt F.

To contact ESRI Technical Support (USA only):
Tel: (909)793-3774 Fax: (909)792-0960 E-mail: support@esri.com

==========================================================================
For additional support use our 24 Hour Support Services:

ESRI's WWW homepage http://www.esri.com
Online Support Center http://support.esri.com
Knowledge Base http://support.esri.com/search/
Discussion Forums http://support.esri.com/forums/
Scripts & Tools http://arcscripts.esri.com/
E-mail Discussion Lists http://support.esri.com/listserve/
ArcObjects Online http://arcobjectsonline.esri.com/
ArcGIS Developer Online http://arcgisdeveloperonline.esri.com
==========================================================================

-----Original Message-----
From: "David Wheelock" <david@oda.state.ok.us>
Sent: Wed, 6 Oct 2004 16:00:21 -0500
To: "ESRI Support" <nimbus@esri.com>
Subject: Re: #388730 (Matt F) winxp ai9.0 spat trying to run a field
process in hydrology modeling tools in spat ana.

Hi, Matt,

Thanks for responding. The grid is 1.4 gig, compressed would be about 850
meg. Is that too big to upload?

I, too, did a small scale test. I cancelled a running fill and then re-ran
it successfully.

Also, I have since done a successful fill on the large dataset that only
took a few hours. Since the hung fill ran for more than half a day, after
I freed up disk space and before I cancelled it, it's clear that there was
something strange going on with that.

I think there are several possibilities for this problem:
1 Bad data.
2 Some artifact of the fact that the fill process hung for lack of disk
space before I cancelled it.
3 Cancelling the fill at a certain very specific point that left some trace
of the original file name behind somewhere. This would be very difficult to
replicate since the fill process has so many steps.

I'm about prepared to treat this as an anomally and move on to the next,
more vexing issue that I haven't been able to get around.

=========================================================
Major Vexing Problem

With the large dataset, I have run a fill, flow direction and flow
accumulation. However, the flow accumulation doesn't have major streams
delineated, just a bunch of disconnected minor stream segments. I've been
through this twice, with exactly the same results.

Additionally, when running the fill and some other processes on this data,
the resulting dataset is the same size in its dimensions and cell size, but
all of the data is compressed into the upper left corner of the dataset to
just a fraction of its original size. Changing the Analysis Cell Size from
"Maximum of Inputs" to "Minimum of Inputs" fixed this, but why should it
make any difference when there's only ONE input raster.

Could the grid be to large? Its dimensions are 9,575 x 37,335.

Interestingly, the grid is a mosaic of 16 pieces downloaded from the USGS
web site. I requested and downloaded the data in two batches of 8 grids on
two different days about a week apart. In both downloads there are areas
that contain no data. However, in the first batch the no-data cells simply
contain no data. In the second batch the no-data cells are given a 0
elevation.

Also, before getting the second batch, I did a mosaic of the first batch of
8, and did the fill, flow direction and flow accumulation and everything was
normal. There was no problem with the flow accumulation.

I've had to pull off at this point because it was taking too much time and I
had other things to do. Also, tomorrow I'm going to a conference, then back
Friday, and next week I will be off, so I won't be able to do much on this
problem till I return on the week of the 18th.

What do you think?

Thanks,
David

----- Original Message ----- From: "ESRI Support" <nimbus@esri.com>
To: "David Wheelock" <david@oda.state.ok.us>
Sent: Tuesday, October 05, 2004 5:38 PM
Subject: Re: #388730 (Matt F) winxp ai9.0 spat trying to run a field process
in hydrology modeling tools in spat ana.





Archive powered by MHonArc 2.6.16.

§