The politics of data
The ancient Greeks considered the town of Delphi holy. According to them it was at the meeting point of the heaven and the earth, where the oracle of the Greek God Apollo guarded over the navel of the earth.
The priestess of the oracle at Delphi was known as the Pythia. Apollo spoke through his oracle, who had to be an older woman of blameless life chosen from among the peasants of the area. The sibyl or prophetess took the name Pythia and sat on a tripod seat over an opening in the earth. When Apollo slew Python, its body fell into this fissure, according to legend, and fumes arose from its decomposing body. Intoxicated by the vapors, the sibyl would fall into a trance, allowing Apollo to possess her spirit. In this state she prophesied. It has been postulated that a gas came out of this opening that is known to produce violent trances, though this theory remains debatable. While in a trance the Pythia "raved" - probably a form of ecstatic speech - and her ravings were "translated" by the priests of the temple into elegant hexameters. People consulted the Delphic oracle on everything from important matters of public policy to personal affairs. (Courtesy: Wikipedia)
Legend has it that people used to gather at the bottom of the hill where Pythia sat; and ask questions to the oracle. The oracle used to answer -- via the medium of Pythia . Nobody had direct access to the actual data. They had to come via the medium.
The point I am trying to establish in today's post is that we've been under the influence of data coming from mystic sources for a long time indeed. Though the form of transport of data has changed; we do not always realize how much of our lives are being controlled by data originating from mediums that translate (or interpret... a more politically correct word) complex wisdom mysteriously and give it to us in convenient packets. Modern branding techniques ensure that the modern equivalent of the Pythia doing this work for us is a person/s of blameless life (to quote the previous Wikipedia description). Companies clamor over each other to pacify us that they had all the research and wherewithal to ensure fairness in the interpretation of data.
The influence of bad data is very insidious. Most of us who are just users do not understand what we've missed out or what was corrupted from the original source. Let me give you one example on how this happens using yet another thought experiment.
Let us say we were stuck in an island with just one cobbler and no shoes. The cobbler would seize the immediate opportunity at hand and get down to business of making shoes for everyone on that island. For doing that, he would be using whatever knowledge that he already had. There will not be any compulsion for him to do any original research... After all, the pressing needs would be a lot more important would be the popular refrain. So the rest of us who do not possess the specialist knowledge of a cobbler, would have to make do with the interpretation that person is making on the subject. In fact, at that point in time; it is quite likely the cobbler himself is blissfully unaware of the deficiencies in the knowledge of his field, and how he may be mis-interpreting it.
Except for the exceptionally curious among us, most of us are content to address data from the outer interface and social expectations of the data locked in a black-box.. When those islanders are wearing the footwear made by that lone cobbler, each of them intuitively know that they want something on their feet...and that the cobbler is providing those. Hence each islander believe they are in possession of sufficient amount of data to get the job done. They are not interested in anything more. Society does not pose any great expectations from the black-box. Hence the black box is never really opened for investigation. The insidious corruption that comes due to this, gets transferred to non-ergonomic footwear.
Now comes the second stage of the problem:
If all the islanders had was that lone cobbler, then maybe things would not have turned out so bad after all. But slowly and surely changes come to the island that necessitate the investigation of the original source of data. For e.g. Competition arrives. There would be others who want to be cobblers. This time around, the cobblers have to fight it out among themselves to capture a limited market. Research starts taking place in the subject. The black box gets opened up, and slowly the cobblers discover some issues that were quietly (or otherwise) hidden away.
At this point, the initial cobbler gets into a fix. Should he now acknowledge that the knowledge which he thought he had actually was corrupted? And that they may have caused flaws in the original designs? If he does, then he will have to produce another kind of footwear. Unfortunately, by the time he realized the deficiencies; the old footwear may have caused so much change in the feet of the wearers, that their feet may have actually got warped. (I've read somewhere that if one wears bad footwear for a long time, the bones inside the feet can even get deformed) Such warped feet would not be able to receive new designs of the footwear; even if research stated that it had its advantages.
I think I'll leave the rest of the story for now. (You can imagine the rest: How the competition may befuddle the issue further by pointing out certain deficiencies convenient to them to usurp the original leader, etc.)
The moral of the story is: Even today, we use handed down data and we don't have the time or inclination to check the sources.
Let us see how this issue rests in the field of architecture. Since 1982 or so when Autodesk (the lone cobbler) came out with Autocad (the footwear we needed) we've been under it's influence. Few manipulations were innocently carried out. Some of them were quite insidious and some were outright manipulations. Our "designing feet" (err...hand) has become so warped, that for many it is perplexing to even imagine a situation that we can talk about design without first talking about CAD drawings (an old form of footwear).
Over time new research and other knowledge influencing the field spoke about possible errors in the field. But will the initial people (the old cobblers) working in this field really change? Will they really admit to their mistakes and redesign the products (the footwear) knowing very well that people who were accustomed to using their old products may find the new ones confusing?
In order to maintain the facade of normalcy, Autodesk and many CAD companies go to great lengths. To date, Autodesk refuses to openly share the data structure used in their DWG file-format. They have locked it away inside a legal black-box. The openDWG consortium often try to reverse-engineer the DWG format and they have succeeded in doing so to a large extent. (This has resulted in litigation with Autodesk) But there are still some unexpected failures of their version of the DWG file-format. So Intellicad, the product based on the openDWG's work do not always work correctly.
Curiously enough the openDWG people themselves try to establish their own hegemony in the field. You have to sign tons of paper work and pay membership fees to get anything useful done.... hmmm. When other companies ("competition of cobblers") came into the field; the same story got repeated.
Someone may point out that fresh research is conducted in this area that filters back to the industry. But if you look at the way those research mechanisms work, you will find intellectual corruption. The typical approach adopted by many research institute is the old dictum of "Publish or Perish" The researchers require funds to attend conferences, run their household as they pore over books/papers. Who will fund all that? The industry itself! So you will find Autodesk or Bentley or one of these companies (the "cobblers") actually sponsoring the researchers and the development of standards. Now, how much of an independence the researcher would have to introduce fresh material is any body's guess.
I have been working in the area of BIM for quite some years, and I can say for sure that many important questions are deliberately sidelined in commercial BIM simply because it does not fit into the overall business workings of the company. (Example: Early stage design, representation of spaces and solids, meta-data representation, etc.) No wonder they say that a camel is a horse designed by a committee.
What is the way out?
I believe the open source movement has shown a way where we can bring in numbers poring over the source of data, and replace the researchers who after being pushed into vulnerable positions are forced to do warped or biased research. Currently there are many data standards committee working on the world wide web. Unfortunately, all the big companies find their own way into those committees and ensure that their data standards are adhered to so that their earlier products are preserved. Hence progress is slow. We do have a long way to go before all of us can have our say. Still, it is a far better situation than before.
The mythical legend of the oracle at Delphi could offer another answer:
The concept of Delphi was used by the American research organization; Rand corporation, to develop a method of arriving at unbiased data. They called it the "Delphi Method" What they proposed is that when a committee works, often the committee members may have personality clashes. To avoid that, the committee works anonymously via an intermediary. The questions and responses to those questions in the committee discussions are made anonymous by the intermediary and passed around. Even the committee members do NOT know who the rest of the members are till the very end i.e. after the conclusion is reached. This ensures equitable participation and interest by everyone. There will not be much personality clash as nobody knows who the others in the committee are and what they have to say. For the "Delphi Method" to work, the committee should be a large one, and the process often takes a long time (because it takes time to make things anonymous )
The Delphi Method has had some quite remarkable successes: For example, It is believed Sony Corporation uses it regularly for their decision making. There are some disadvantages of the Delphi Method. One of them being that the intermediary is known to everyone in the committee. However, I believe that even that can be made anonymous if we intelligently use the Internet to drive a standards committee using the Delphi Method. Read more here I personally believe that instead of setting up closed committees that work on world standards which may get populated by vested interests (Most of the w3 committees have people from commercial organizations actively working in them) it may be better to have mechanisms to collect democratic feedback using a modified Delphi process from as many people who care to participate.
God is in the details said Mies. The computer guys know that Devil is in the Data. Raw data is at the root of a food chain of knowledge. Data gets transformed to information. Thence to knowledge and lastly it resides as wisdom in the minds of a few enlightened ones. So data standards are very important. Whoever who can manipulate and control data standards can control the future course of action in that field. The rest of us merely gather at the foot of the hill where such Pythias reside, bow in front of such large corporations and take whatever that is given to us.
The priestess of the oracle at Delphi was known as the Pythia. Apollo spoke through his oracle, who had to be an older woman of blameless life chosen from among the peasants of the area. The sibyl or prophetess took the name Pythia and sat on a tripod seat over an opening in the earth. When Apollo slew Python, its body fell into this fissure, according to legend, and fumes arose from its decomposing body. Intoxicated by the vapors, the sibyl would fall into a trance, allowing Apollo to possess her spirit. In this state she prophesied. It has been postulated that a gas came out of this opening that is known to produce violent trances, though this theory remains debatable. While in a trance the Pythia "raved" - probably a form of ecstatic speech - and her ravings were "translated" by the priests of the temple into elegant hexameters. People consulted the Delphic oracle on everything from important matters of public policy to personal affairs. (Courtesy: Wikipedia)
Legend has it that people used to gather at the bottom of the hill where Pythia sat; and ask questions to the oracle. The oracle used to answer -- via the medium of Pythia . Nobody had direct access to the actual data. They had to come via the medium.
The point I am trying to establish in today's post is that we've been under the influence of data coming from mystic sources for a long time indeed. Though the form of transport of data has changed; we do not always realize how much of our lives are being controlled by data originating from mediums that translate (or interpret... a more politically correct word) complex wisdom mysteriously and give it to us in convenient packets. Modern branding techniques ensure that the modern equivalent of the Pythia doing this work for us is a person/s of blameless life (to quote the previous Wikipedia description). Companies clamor over each other to pacify us that they had all the research and wherewithal to ensure fairness in the interpretation of data.
The influence of bad data is very insidious. Most of us who are just users do not understand what we've missed out or what was corrupted from the original source. Let me give you one example on how this happens using yet another thought experiment.
Let us say we were stuck in an island with just one cobbler and no shoes. The cobbler would seize the immediate opportunity at hand and get down to business of making shoes for everyone on that island. For doing that, he would be using whatever knowledge that he already had. There will not be any compulsion for him to do any original research... After all, the pressing needs would be a lot more important would be the popular refrain. So the rest of us who do not possess the specialist knowledge of a cobbler, would have to make do with the interpretation that person is making on the subject. In fact, at that point in time; it is quite likely the cobbler himself is blissfully unaware of the deficiencies in the knowledge of his field, and how he may be mis-interpreting it.
Except for the exceptionally curious among us, most of us are content to address data from the outer interface and social expectations of the data locked in a black-box.. When those islanders are wearing the footwear made by that lone cobbler, each of them intuitively know that they want something on their feet...and that the cobbler is providing those. Hence each islander believe they are in possession of sufficient amount of data to get the job done. They are not interested in anything more. Society does not pose any great expectations from the black-box. Hence the black box is never really opened for investigation. The insidious corruption that comes due to this, gets transferred to non-ergonomic footwear.
Now comes the second stage of the problem:
If all the islanders had was that lone cobbler, then maybe things would not have turned out so bad after all. But slowly and surely changes come to the island that necessitate the investigation of the original source of data. For e.g. Competition arrives. There would be others who want to be cobblers. This time around, the cobblers have to fight it out among themselves to capture a limited market. Research starts taking place in the subject. The black box gets opened up, and slowly the cobblers discover some issues that were quietly (or otherwise) hidden away.
At this point, the initial cobbler gets into a fix. Should he now acknowledge that the knowledge which he thought he had actually was corrupted? And that they may have caused flaws in the original designs? If he does, then he will have to produce another kind of footwear. Unfortunately, by the time he realized the deficiencies; the old footwear may have caused so much change in the feet of the wearers, that their feet may have actually got warped. (I've read somewhere that if one wears bad footwear for a long time, the bones inside the feet can even get deformed) Such warped feet would not be able to receive new designs of the footwear; even if research stated that it had its advantages.
I think I'll leave the rest of the story for now. (You can imagine the rest: How the competition may befuddle the issue further by pointing out certain deficiencies convenient to them to usurp the original leader, etc.)
The moral of the story is: Even today, we use handed down data and we don't have the time or inclination to check the sources.
Let us see how this issue rests in the field of architecture. Since 1982 or so when Autodesk (the lone cobbler) came out with Autocad (the footwear we needed) we've been under it's influence. Few manipulations were innocently carried out. Some of them were quite insidious and some were outright manipulations. Our "designing feet" (err...hand) has become so warped, that for many it is perplexing to even imagine a situation that we can talk about design without first talking about CAD drawings (an old form of footwear).
Over time new research and other knowledge influencing the field spoke about possible errors in the field. But will the initial people (the old cobblers) working in this field really change? Will they really admit to their mistakes and redesign the products (the footwear) knowing very well that people who were accustomed to using their old products may find the new ones confusing?
In order to maintain the facade of normalcy, Autodesk and many CAD companies go to great lengths. To date, Autodesk refuses to openly share the data structure used in their DWG file-format. They have locked it away inside a legal black-box. The openDWG consortium often try to reverse-engineer the DWG format and they have succeeded in doing so to a large extent. (This has resulted in litigation with Autodesk) But there are still some unexpected failures of their version of the DWG file-format. So Intellicad, the product based on the openDWG's work do not always work correctly.
Curiously enough the openDWG people themselves try to establish their own hegemony in the field. You have to sign tons of paper work and pay membership fees to get anything useful done.... hmmm. When other companies ("competition of cobblers") came into the field; the same story got repeated.
Someone may point out that fresh research is conducted in this area that filters back to the industry. But if you look at the way those research mechanisms work, you will find intellectual corruption. The typical approach adopted by many research institute is the old dictum of "Publish or Perish" The researchers require funds to attend conferences, run their household as they pore over books/papers. Who will fund all that? The industry itself! So you will find Autodesk or Bentley or one of these companies (the "cobblers") actually sponsoring the researchers and the development of standards. Now, how much of an independence the researcher would have to introduce fresh material is any body's guess.
I have been working in the area of BIM for quite some years, and I can say for sure that many important questions are deliberately sidelined in commercial BIM simply because it does not fit into the overall business workings of the company. (Example: Early stage design, representation of spaces and solids, meta-data representation, etc.) No wonder they say that a camel is a horse designed by a committee.
What is the way out?
I believe the open source movement has shown a way where we can bring in numbers poring over the source of data, and replace the researchers who after being pushed into vulnerable positions are forced to do warped or biased research. Currently there are many data standards committee working on the world wide web. Unfortunately, all the big companies find their own way into those committees and ensure that their data standards are adhered to so that their earlier products are preserved. Hence progress is slow. We do have a long way to go before all of us can have our say. Still, it is a far better situation than before.
The mythical legend of the oracle at Delphi could offer another answer:
The concept of Delphi was used by the American research organization; Rand corporation, to develop a method of arriving at unbiased data. They called it the "Delphi Method" What they proposed is that when a committee works, often the committee members may have personality clashes. To avoid that, the committee works anonymously via an intermediary. The questions and responses to those questions in the committee discussions are made anonymous by the intermediary and passed around. Even the committee members do NOT know who the rest of the members are till the very end i.e. after the conclusion is reached. This ensures equitable participation and interest by everyone. There will not be much personality clash as nobody knows who the others in the committee are and what they have to say. For the "Delphi Method" to work, the committee should be a large one, and the process often takes a long time (because it takes time to make things anonymous )
The Delphi Method has had some quite remarkable successes: For example, It is believed Sony Corporation uses it regularly for their decision making. There are some disadvantages of the Delphi Method. One of them being that the intermediary is known to everyone in the committee. However, I believe that even that can be made anonymous if we intelligently use the Internet to drive a standards committee using the Delphi Method. Read more here I personally believe that instead of setting up closed committees that work on world standards which may get populated by vested interests (Most of the w3 committees have people from commercial organizations actively working in them) it may be better to have mechanisms to collect democratic feedback using a modified Delphi process from as many people who care to participate.
God is in the details said Mies. The computer guys know that Devil is in the Data. Raw data is at the root of a food chain of knowledge. Data gets transformed to information. Thence to knowledge and lastly it resides as wisdom in the minds of a few enlightened ones. So data standards are very important. Whoever who can manipulate and control data standards can control the future course of action in that field. The rest of us merely gather at the foot of the hill where such Pythias reside, bow in front of such large corporations and take whatever that is given to us.
Comment
-
Comment by Himadri Mayank on May 25, 2009 at 10:51pm - Ok. Agreed. It is tough to agree without a debate.
-
Comment by Sabu Francis on May 16, 2009 at 12:07am - The "day open source would be more acceptable to the users" can be proved as one types on an IBM PC compatible computer!
-
Comment by Sabu Francis on May 16, 2009 at 12:04am
-
This is a very common misconception: That open source people are philanthropists and that they have plenty of rough edges that needs to be ironed out, etc. But the truth is: That is where the money really is. Google gives away almost everything free. Are they dumb? Do they stop making money because of that? Will they make MORE money if they started charging for their services? In fact they will drastically reduce their money if they really started charging. The IBM guys were not someone who did not think through their strategy when they open-sourced their hardware. They were as hard-nosed as the next guy.
Neither is open-source an 'imposition of socialism' ! I can't figure out can that leap be made by any stretch. It is as clear-cut a free-market economy as the best (or I would say it is the way the free-market economy will have to go)
-
Comment by Himadri Mayank on May 15, 2009 at 1:52pm
-
Oh yes, I agree with most of the stuff being said here for Open Source as a medium of acheiving better efficient solutions. And ideally, that should be. However, when you see champions of open source (we had a chance to listen to Richard Stallman at IIT) against a kind of Steve Jobs or Bill Gates, what you immediately notice is the 'difference in packaging'. And to a lot of people, that delivers. I don't see Open Source gathering a mass momentum without packaging it in markets. If Blender, Wikipedia, Apache and BitTorrent are not holding onto their 'intellectual properties', maybe they are too philanthropic or possibly they do not foresee a monopoly in their market segment. A few saints can't make all the merchants otherworldly.
What I essentially do not foresee is the 'imposition of socialism' to a 'capitalist software market'. And if one feels that only open source would eventually survive, let the others die out. The day open source would be more acceptable to the users, markets would themselves force the non-performers to die.
Sabu, to read your posts are a delight. Keep posting.
-
Comment by Sabu Francis on May 3, 2009 at 10:18am
- I again reiterate: It is difficult to intuit some of the things that happen on the Internet. Even the concept of innovation is undergoing err....an innovation. The scientists talk about something "emergence" ... Just the way how termites collectively can innovate and make a grand air-conditioned 'edifice' for themselves even though each tiny termite is not intelligent enough to hold that spark to create such structures, in a similar fashion; I believe it should be possible for humans too collectively come together and defeat erstwhile hegemonies. BTW, open source as a concept has been seen even in hardware. One of the most striking ones being the IBM Personal Computer (Not many talk about this as an 'open source') Computer manufacturers were completed stupefied when IBM decided to 'open up' the design of the PC. When it came in 1980/81, everything about it was open; including the bios. If I remember right, Phoenix was the first Taiwanese company that cloned the IBM bios just because the internals was kept open by IBM with no copyright restrictions. Earlier to that, computer companies used to hang on to their 'edge' by making incompatible systems. Now, where are all those companies? (DEC, for e.g.) They all died out!
-
Comment by Vishal Charles on May 3, 2009 at 9:13am -
Open source and copyrighted (paid) debate has a long history. Hardware makers supported free software movement and software makers wanted to get paid for their work. I'm a big fan of open source but it's curious to see how it's related to the human psyche. anything virtual is fair game but we hold different standards for tangible goods. It's easy to pay for something you can hold in your hands. I like the open source idea that involves collaboration on a much larger scale. I can imagine downsides of this approach too! The spark of genius is not really a community thing IMO.
On a curious note, the open source movement is essentially computer professionals/ hobbyists working together to develop solutions for their own industry. It's easy to imagine the excitement when you make tools for your own use. You know what you want and you know the stuff you need to develop that tool. Architects are architects. Sometimes I think hoping for a great open source BIM tool is wishful thinking. How many architects are there with the skill, time or inclination for this effort ?
-
Comment by Sabu Francis on May 2, 2009 at 4:14am
-
What works in the 'real' world does not work necessarily on the Internet. In fact, usually it works quite differently. You may be assuming that profits can achieved only in one manner. That is an archaic approach. If we did not have the Internet, we had probably no option but to hold on to intellectual properties (formula 'x'). And why is that? Because there was a clear differentiation between the innovator and the users of the innovation. Not so with the Internet. There, everyone can participate. This has upset many intuitions (one of them being the concept of profits) Take the development of Linux or Apache. If you see http://news.netcraft.com/ you'll notice that of the top 10 reliable hosts, only one is running a Microsoft server.Everything else is free. If you take a look at the graphs, Apache is way ahead of every other server being used on the net.
Marketing concepts like the 'long tail' concept (www.thelongtail.com) cannot be done using the 'real' world. But that is exactly what can work on the net.
It is the big corporations who has probably sowed the myth that profits are exactly the way they were prior to the Internet era. We must remember that the Internet is hardly 20-25 years old: an infant historically. So there is much maelstrom and confusion in these times, which these large companies exploit. They sow the meme that no other definition of profitability is possible.
I believe the concept of 'profitability' now includes the portion one has to pay for a better future. Not just the money one makes for oneself. It includes stakeholder wealth, not just shareholder wealth. Though it may sound idealistic, but that is a very practical definition. With this new definition, one has to go beyond conventional methods and the Internet does makes it possible. A lot of companies (Blender, Wikipedia, Apache, BitTorrent,...) are making tons of money without holding onto 'intellectual properties' and they are also contributing to stakeholder wealth. Data standards lobbies should not be looked upon as mere "oh, it is expected they'll protect their interest" and be resigned to that.
-
Comment by Himadri Mayank on May 2, 2009 at 1:50am
-
Sabu, looks like you have taken on the monopolistic or rather oligopolistic market of CAD and BIM softwares. Although what you say is correct, I think till the time curious people (the OpenDWGs) come up with a striking solution which rattles the monopoly of Autodesk, it would be hilarious for the curious people to expect that Autodesk would give its code. It makes money for them.
It is similar to asking Coke or Pepsi about their "x" formula for the beverage. Of course if the "x" formula was known, many housewives would have improvised over it to give better tasting beverages. But, it earns money for the beverage companies and they have every right to own it.
Having said that, I believe, market is not in the interest of the people but for profits. When the curious people stop hacking the ways of the lone cobbler and devise their own ways of providing shoes to the poeple of the island, it would be for profit and people would get better shoes.
About data standards, everyone is entitled to lobby for their own standard, and the corporations would do that, because it is for profits. If the curious people can make a better product, a better data standard than Autodesk, it is just time that people would start using that and the Corporations would have to rethink their strategy of monopoly, as it would hurt their profits.
© 2010 Created by Archisage
You need to be a member of Archisage - Architecture Social to add comments!
Join Archisage - Architecture Social