Data, again

Many thanks to Daniel, who apparently knows the magical incantations necessary to transform ESRI shapefile data into XML files. If you look in
http://www.cs.uoregon.edu/classes/08F/cis422/data/OSBxml/
you will find the full collection as osb_gis_xml.tar.zip or you can look in the osb_gis_xml directory to get individual xml files.

It's very late, and I understand if you are not able to incorporate this into your project before the deadline. On the other hand, it would be great if at least some of you could do so, or at least give it a try and document what problems need to be solved to make it work.

Update: I've looked through the XML data and I'm not so sure we're making progress. As near as I can tell, what we have is "PolygonB" objects that are represented as byte arrays (and I don't know how to interpret those byte arrays). It seems that "PolygonN" objects have a field called "Rings", which are arrays of (x,y) coordinate pairs, and that's what we were expecting. I don't know how to turn "PolygonB" objects into "PolygonN" objects. Maybe someone does?

OSB data? Please have a look

I have an XML file from Amy. It's one big XML file, rather than a file per layer, but maybe that's not a problem (??). As I eyeball it, though, it seems to me like the shapes are defined in a different way than in the campus map XML files ... it looks like there is a sort of hex encoding of sequences of coordinates, instead of XML-ish encoding of each coordinate pair. So, my best guess is that this will not work with the "generalized" input modules that you have worked on. But please give it a try, and let me know what you discover. Is there some obvious way to transform this XML into something we can deal with?

I also have the raw ESRI shapefile data, if anyone knows the ArcGIS program well enough to manage the XML export from that. (I don't, but I am beginning to see that it's something I had better learn.)

The XML file is at: http://www.cs.uoregon.edu/classes/08F/cis422/data/OSBdata.xml

Midterm grading progress

I've made a first pass through all the midterms now, and assigned tentative scores ... but I want to make a second pass through to make sure your scores don't depend too much on how warm my coffee was or what time of day it was when I read yours. I will probably need to "bin" them (group into rough equivalence classes given the same score) to avoid too much random variation in scores.

Generally I'm pretty pleased. Most of the midterms did draw from both papers, and thoughtfully applied them to the specific project descriptions. A few of the midterms were particularly creative or insightful. Some were a bit generic, describing only general goals ("encourage open communication") rather than concrete plans for meeting those goals. Many, I thought, underestimated the difficulty of precisely defining interfaces and reaching a common understanding of what Jackson calls "ground terms". That's understandable, and I think it's hard to grasp how slippery terms and interface definitions can be until you've had some (possibly unpleasant) experience working with others across organizational boundaries.

Midterm

There is just one question on the midterm, but it asks you to draw from both of the papers we read and discussed. (You can also draw from your project experience, but be sure to draw from the papers as well.)

The midterm is due at 5pm Wednesday, November 12. Turn it in by sending your answer as plain text in the body of an email message, using this link:
Turnin email

Your answer should be 500 words or less. (The question is 331 words, so the upper limit is about 1.5 times as long as the question.)

Here is the question:

Development project teams may be distributed between organizations for a variety of reasons. One of the reasons is to combine domain expertise from different organizations participating in a collaborative project. Here are two hypothetical* examples:

a) Nike and Apple are jointly developing "smart shoe" systems for runners. Sensors in the Nike shoes will interact with iPod music players designed and manufactured by Apple. The adaptor between the shoes and the music players will be jointly designed and implemented by a team composed of a shoe sensor design sub-team at Nike headquarters in Portland; a web interface team at Apple's headquarters in Sunnyvale; and a data acquisition sub-team based in Apple's Portland offices. Each sub-team includes experts in one part of the system.

b) Electrical Geodesics, Inc. is a small Eugene company that produces dense-array electroencephalograph (dEEG) devices for measuring brain activity. These dEEG devices can be used together with functional magnetic resonance imaging (fMRI) devices to obtain more complete data about brain functioning. Combining and visualizing the large data sets produced by these devices requires advanced image-processing algorithms running on parallel computers. A project team to build a prototype dEEG+fMRI combined visualization system is composed of a dEEG expert from EGI, an expert in neuroanatomy and an expert in fMRI imaging from the UO psychology department, a computer science professor, and three graduate students, one from psychology and two from computer science.

Pick either one of these examples for your answer. Suppose you have been asked to help organize and manage that example project. How will you apply lessons from the papers we read? Please draw at least some from both papers. Don't just repeat advice from those papers --- explain how you would apply it to the specific example you chose.

* Parts of these examples are pure imagination, but both are (as they say in the movies) "inspired by a true story". See:
http://www.businessweek.com/technology/content/may2006/tc20060523_569911.htm
http://www.egi.com/research-division-converging-neurotechnologies
I have no idea about the actual composition and location of the development teams.

Reading: Jackson

The second paper I will ask you to read concerns requirements analysis and specification:

Jackson, M. 1995. The world and the machine. In Proceedings of the 17th international Conference on Software Engineering (Seattle, Washington, United States, April 24 - 28, 1995). ICSE '95. ACM, New York, NY, 283-292. DOI= http://doi.acm.org/10.1145/225014.225041

This is among the most lucid accounts I know of what it means to write a specification. The writing is light-hearted (the paragraph about steering mechanisms for cars busts me up laughing), but it's a very serious and deep consideration of the topic ... it's worth reading more than once, and thinking carefully about.

The paper accompanied an invited (keynote) talk at the International Conference on Software Engineering. Two years later, Jackson and Zave published a journal paper which encompasses some of the same material. If you like this paper, you might (at your option) follow on with:

Zave, P. and Jackson, M. 1997. Four dark corners of requirements engineering. ACM Trans. Softw. Eng. Methodol. 6, 1 (Jan. 1997), 1-30. DOI= http://doi.acm.org/10.1145/237432.237434

Some test data, but ...

Amy exported some XML from a City of Eugene geodatabase for me, and I've put it in an accessible place, but I'm not sure how useful it will be. It looks like the individual layer files are basically empty of actual shape data. The full XML dump of the database is quite large (53M), but I'm not sure if it contains data that corresponds to layer data in the campus map. If you want to give it a try, it's at
/home/faculty/michal/public_html/08F-GIS-Eugene
on the Sun (ix) file system, or
http://www.cs.uoregon.edu/~michal/08F-GIS-Eugene
through the web server.

The .shx files are in a native ESRI data format, and the .dbf files are some kind of native database file ... the Unix "file" command claims they are in DBase 3 data file format, which is possible. The "file" command claims the .mdb file is a Microsoft Access database, but that seems unlikely. If there is data useful for testing, it is likely to be in the file XMLExport_GeoDataBase.xml . Warning, that's 53 megabytes of XML text with really long lines. Whether it has anything usable or not, I really can't tell ... trying to look through it with Emacs didn't get me very far, and it crashed Firefox!

We're hoping for real OSB data at the end of the week.

Why Keith's parser doesn't produce an XML file

A few people have been confused about how Keith Albin's stylesheet parser fits with the rest of the system. In particular, there is a "Main" program in the parser directory, but that "Main" program is just a test harness to make sure the parser can accept the grammar of the style sheet. Unlike the older stylesheet parser (used by the Red and Blue teams), Keith's stylesheet parser does not produce an intermediate XML file representation of the style sheet.

So what does it do? If you call it from that Main class, it does basically nothing ... it builds some internal data structures and then throws them away. But take a look at StyleSheetReader.java in the actionscript directory --- this is where the parser is really called. It builds those data structures in the readStyles method, and then uses the data structures in the applyStyles method.

I don't know this code inside-out, but I think the basic logic goes something like this: We read the style sheet and create some internal tables with (keyword, value) pairs (which may actually be triples ... there is something called a "modifier" and I'm not certain what that is). Then we read all the map "shapes" from an XML file. Then, for each of those shapes, applyStyles looks in the tables and figures out what attributes should apply, and sticks those attributes into the shape object. Then we generate code (actionscript or a data structure for the Java display engine, as the case may be) using the shape information and all its style attributes.