[cryptome] Re: NYC Taxicab Log Dump

  • From: doug <douglasrankine2001@xxxxxxxxxxx>
  • To: cryptome@xxxxxxxxxxxxx
  • Date: Thu, 26 Jun 2014 20:31:57 +0100

Dear Aftermath...or is it Bill,
One of the things that I learned early in life, particularly when I moved to London and then to the Isle of Wight, was that taxi drivers are a mine of information on the local environment. When one is contemplating moving into a new area, then what can be better than entering into conversation with one's taxi driver to find out who is who and who does what in the local environment. Of course, one doesn't have to enter into conversation with one's cabbie, after all, one is a captive audience, one just has to guide the conversation with a few "open" questions which helps to ascertain the kind of environment and type of people one is going or hoping to meet up with if one settles there. The nice thing about such open source messages and conversations is that they, up until now, is that they were private and not recorded; and can establish so much of what one wants to know, without any additional expense rather than the hire. Of course, nowadays, the metadata is just as important...(if not more so)...than the content of the conversation, but that is the internet and all this new information technology for you.
When I visited New York for a holiday...before 9/11, I met up with a cabbie whose English was very poor. He was a Russian immigrant, yet knew his way around New York so well, and was able to tell me and me missus the best places to go and which places to avoid and at what time. All it cost was a nice tip at the end of the journey, with which he was very pleased, and he could see that us Scots were not at all prejudiced about his origins or his politics. What more can one ask for in life...sarcasm is such an ugly word...to be accused of using it is so hurtful...;-) . 
ATB
Dougie.
P.S. Have you ever read any of Le Carre's novels? He has written 23 and like all good security and intelligence buffs, I have read, nay, studied all of them. You can find them in all good bookshops and they are available on line very cheap. Like le Carre, I find fiction far more educational than non-fiction. And fiction has the added advantage that it is not classified or subject to secrecy laws, yet contains all the craft of the trade of information. Beware of course, that le Carre's main theme, as is that of William Boyd, is betrayal... 

On 26/06/14 19:54, Aftermath wrote:

... cant tell if sarcasm or not..?
but yeah, I just found the link and I thought this list would appreciate the share :)
On Thu, Jun 26, 2014 at 11:48 AM, doug <douglasrankine2001@xxxxxxxxxxx <mailto:douglasrankine2001@xxxxxxxxxxx>> wrote: 

    On 26/06/14 18:19, Aftermath wrote:
    Dear Aftermath...or is it Bill?

    I thought this posting was absolutely brilliant. My
    congratulations to you for bringing it to our attention and Mr.
    Wong wh did the initial research.  Why?  Because...if you think on
    it, you have saved the NSA, the CIA and the intelligence and
    security departments  of the New York Police Force huge amounts of
    money on expenditure of resources...and foreign organisations such
    as the Chinese, the Israelis, the Russians and any ole Tom, Dick
    and Harry of a security organisation a lot of time.  Even the
    United Nations security and intelligence services will be all agog
    at your posting when they read it.  I hope you are not claiming
    copyright on it...;-) . At the click of a button, they can all
    find out  who has been doing what in New York....United Nations
    officials, Ambassadors, envoys, consuls and all sorts of public
    and civil service officials, secret service, intelligence
    services, even the private sector.  All this metadata available to
    link up with known associations and links...and all open source
    too, who visited who and when, who was where and when, with a
    little help from the smart mobile phone, the most valuable source
    of i.d. location and contacts, you have provided the world, with a
    wealth of information.  Just think...the amount of money you have
    saved the world's intelligence organisations.  Absolutely f*cking
    brilliant....My congratulations...
    ATB;-)
    Dougie.
    P.S. Who needs privacy and encryption when we have all this stuff
    about the world leaders in our very own hands.  It is all a
    question of pressing the right button...


        http://chriswhong.com/open-data/foil_nyc_taxi/
        and
        https://medium.com/@vijayp/f6bc289679a1


        from the second link...

        "
        Recently, thanks to a Freedom of Information request, Chris
        Whongreceived and made public a complete dump of historical
        trip and fare logs from NYC taxis. It’s pretty incredible:
        there are over 20GB of uncompressed data comprising more than
        173 million individual trips. Each trip record includes the
        pickup and dropoff location and time, anonymized hack licence
        number and medallion number (i.e. the taxi’s unique id number,
        3F38, in my photo above), and other metadata.

        These data are a veritable trove for people who love cities,
        transit, and data visualization. But there’s a big problem:
        the personally identifiable information (the driver’s licence
        number and taxi number) hasn’t been anonymized properly — 
        what’s worse, it’s trivial to undo, and with other publicly
        available data, one can even figure out which person drove
        each trip. In the rest of this post, I’ll describe the
        structure of the data, what the person/people who released the
        data did wrong, how easy it is to deanonymize, and the lessons
        other agencies should learn from this. (And yes, I’ll also
        explain how rainbows fit in).


        The NYC taxi data consist of a number of CSV-files.....
        "

Other related posts:

  1. Home
  2. cryptome
  3. Archive
  4. 06-2014