[cryptome] Re: NYC Taxicab Log Dump

  • From: Aftermath <aftermath.thegreat@xxxxxxxxx>
  • To: "cryptome@xxxxxxxxxxxxx" <cryptome@xxxxxxxxxxxxx>
  • Date: Thu, 26 Jun 2014 13:06:19 -0700

OK :) and no, Ill have to look into Le Carre's novels.

And I prefer William ;)

On Thu, Jun 26, 2014 at 12:31 PM, doug <douglasrankine2001@xxxxxxxxxxx>

>  Dear Aftermath...or is it Bill,
> One of the things that I learned early in life, particularly when I moved
> to London and then to the Isle of Wight, was that taxi drivers are a mine
> of information on the local environment.  When one is contemplating moving
> into a new area, then what can be better than entering into conversation
> with one's taxi driver to find out who is who and who does what in the
> local environment.  Of course, one doesn't have to enter into conversation
> with one's cabbie, after all, one is a captive audience, one just has to
> guide the conversation with a few "open" questions which helps to ascertain
> the kind of environment and type of people one is going or hoping to meet
> up with if one settles there.  The nice thing about such open source
> messages and conversations is that they, up until now, is that they were
> private and not recorded; and can establish so much of what one wants to
> know, without any additional expense rather than the hire. Of course,
> nowadays, the metadata is just as important...(if not more so)...than the
> content of the conversation, but that is the internet and all this new
> information technology for you.
> When I visited New York for a holiday...before 9/11, I met up with a
> cabbie whose English was very poor.  He was a Russian immigrant, yet knew
> his way around New York so well, and was able to tell me and me missus the
> best places to go and which places to avoid and at what time.  All it cost
> was a nice tip at the end of the journey, with which he was very pleased,
> and he could see that us Scots were not at all prejudiced about his origins
> or his politics.  What more can one ask for in life...sarcasm is such an
> ugly word...to be accused of using it is so hurtful... ;-) .
> Dougie.
> P.S. Have you ever read any of Le Carre's novels?  He has written 23 and
> like all good security and intelligence buffs, I have read, nay, studied
> all of them.  You can find them in all good bookshops and they are
> available on line very cheap.  Like le Carre, I find fiction far more
> educational than non-fiction.  And fiction has the added advantage that it
> is not classified or subject to secrecy laws, yet contains all the craft of
> the trade of information.  Beware of course, that le Carre's main theme, as
> is that of William Boyd, is betrayal...
> On 26/06/14 19:54, Aftermath wrote:
> ... cant tell if sarcasm or not..?
>  but yeah, I just found the link and I thought this list would appreciate
> the share :)
>  On Thu, Jun 26, 2014 at 11:48 AM, doug <douglasrankine2001@xxxxxxxxxxx>
> wrote:
>> On 26/06/14 18:19, Aftermath wrote:
>> Dear Aftermath...or is it Bill?
>> I thought this posting was absolutely brilliant. My congratulations to
>> you for bringing it to our attention and Mr. Wong wh did the initial
>> research.  Why?  Because...if you think on it, you have saved the NSA, the
>> CIA and the intelligence and security departments  of the New York Police
>> Force huge amounts of money on expenditure of resources...and foreign
>> organisations such as the Chinese, the Israelis, the Russians and any ole
>> Tom, Dick and Harry of a security organisation a lot of time.  Even the
>> United Nations security and intelligence services will be all agog at your
>> posting when they read it.  I hope you are not claiming copyright on
>> it...;-) . At the click of a button, they can all find out  who has been
>> doing what in New York....United Nations officials, Ambassadors, envoys,
>> consuls and all sorts of public and civil service officials, secret
>> service, intelligence services, even the private sector.  All this metadata
>> available to link up with known associations and links...and all open
>> source too, who visited who and when, who was where and when, with a little
>> help from the smart mobile phone, the most valuable source of i.d. location
>> and contacts, you have provided the world, with a wealth of information.
>>  Just think...the amount of money you have saved the world's intelligence
>> organisations.  Absolutely f*cking brilliant....My congratulations...
>> ATB;-)
>> Dougie.
>> P.S. Who needs privacy and encryption when we have all this stuff about
>> the world leaders in our very own hands.  It is all a question of pressing
>> the right button...
>>  http://chriswhong.com/open-data/foil_nyc_taxi/
>>> and
>>> https://medium.com/@vijayp/f6bc289679a1
>>> from the second link...
>>> "
>>> Recently, thanks to a Freedom of Information request, Chris
>>> Whongreceived and made public a complete dump of historical trip and fare
>>> logs from NYC taxis. It’s pretty incredible: there are over 20GB of
>>> uncompressed data comprising more than 173 million individual trips. Each
>>> trip record includes the pickup and dropoff location and time, anonymized
>>> hack licence number and medallion number (i.e. the taxi’s unique id number,
>>> 3F38, in my photo above), and other metadata.
>>> These data are a veritable trove for people who love cities, transit,
>>> and data visualization. But there’s a big problem: the personally
>>> identifiable information (the driver’s licence number and taxi number)
>>> hasn’t been anonymized properly — what’s worse, it’s trivial to undo, and
>>> with other publicly available data, one can even figure out which person
>>> drove each trip. In the rest of this post, I’ll describe the structure of
>>> the data, what the person/people who released the data did wrong, how easy
>>> it is to deanonymize, and the lessons other agencies should learn from
>>> this. (And yes, I’ll also explain how rainbows fit in).
>>> The NYC taxi data consist of a number of CSV-files.....
>>> "

Other related posts: