[cryptome] Re: NYC Taxicab Log Dump

  • From: Aftermath <aftermath.thegreat@xxxxxxxxx>
  • To: "cryptome@xxxxxxxxxxxxx" <cryptome@xxxxxxxxxxxxx>
  • Date: Thu, 26 Jun 2014 14:42:13 -0700

I will look into those..

I have not read Puzzle Place, but I read Body of Secrets by the same
author. Deeply fascinating


On Thu, Jun 26, 2014 at 1:30 PM, doug <douglasrankine2001@xxxxxxxxxxx>

>  Excellent...I am very pleased to have added to your depth and breadth of
> education...as you have added to mine. And I am very pleased that we hold
> at least one author in common. I am always grateful when someone adds to my
> education or makes me think or act in a new way.
> Can I also suggest a book by Norman Mailer...a bit dated now, and of
> course he died the other year without writing the sequel.  It is called
> "Harlot's Ghost", available at all good bookshops...and a used edition is
> very cheap at Amazon...if I am allowed to advertise such things here on the
> list...though one can also get it on hire from ones local library for
> free...if there is such an organisation these days.
>   In the book you will discover a number of things about the CIA and such
> like.  How the organisation is run, the compartmentalisation and how it
> works for and against the efficiency of the organisation and how it
> protects those who don't want to be brought out into the heat, the names of
> the families who were involved in its creation, their ancestors and
> descendants who are still involved (pseudonyms of course) and such like.
> Mailer spent many years researching it, and it is a bit dated now, but the
> essence is still there and it is a fascinating story.  Puzzle Palace was my
> first introduction to the intelligence game and after reading it, I was
> hooked.
> Also, there is something in there which helps all good spies...or dare I
> say, "spy buffs" in the book to memorise things like phone numbers and
> various other kinds of numbers, data and personal information and such
> stuff.  None of this stuff is new of course, neither is it classified, and
> one can find a plethora of websites on line which offer various methods of
> improving one's memory, though one should never make the mistake that such
> improvement attaches to ones intellect and also, like many aspects of
> cryptography and security, there is a lot of snake oil around, which costs
> a lot of money.  I am one of those who believes that mind mapping and the
> use of memory techniques can save one a lot of time, as well as helping
> with assimilation and retention of information; but I have never been able
> to make up my mind whether they in fact do so, or are just a trick to get
> one to spend some money.  I am a great believer that knowledge should be
> free and open to everyone who cares to educate themselves.
>  Mind you the Ancient Greeks and Romans used such methods, and there are a
> number of biographies which suggest that great leaders used such skills to
> remember things like battle plans or complex algorithms or even the names
> of every soldier in their armies, such as Alexander...however, he died
> early, so such knowledge didn't save him.
> On 26/06/14 21:06, Aftermath wrote:
> OK :) and no, Ill have to look into Le Carre's novels.
>  And I prefer William ;)
> On Thu, Jun 26, 2014 at 12:31 PM, doug <douglasrankine2001@xxxxxxxxxxx>
> wrote:
>>  Dear Aftermath...or is it Bill,
>> One of the things that I learned early in life, particularly when I moved
>> to London and then to the Isle of Wight, was that taxi drivers are a mine
>> of information on the local environment.  When one is contemplating moving
>> into a new area, then what can be better than entering into conversation
>> with one's taxi driver to find out who is who and who does what in the
>> local environment.  Of course, one doesn't have to enter into conversation
>> with one's cabbie, after all, one is a captive audience, one just has to
>> guide the conversation with a few "open" questions which helps to ascertain
>> the kind of environment and type of people one is going or hoping to meet
>> up with if one settles there.  The nice thing about such open source
>> messages and conversations is that they, up until now, is that they were
>> private and not recorded; and can establish so much of what one wants to
>> know, without any additional expense rather than the hire. Of course,
>> nowadays, the metadata is just as important...(if not more so)...than the
>> content of the conversation, but that is the internet and all this new
>> information technology for you.
>> When I visited New York for a holiday...before 9/11, I met up with a
>> cabbie whose English was very poor.  He was a Russian immigrant, yet knew
>> his way around New York so well, and was able to tell me and me missus the
>> best places to go and which places to avoid and at what time.  All it cost
>> was a nice tip at the end of the journey, with which he was very pleased,
>> and he could see that us Scots were not at all prejudiced about his origins
>> or his politics.  What more can one ask for in life...sarcasm is such an
>> ugly word...to be accused of using it is so hurtful... ;-) .
>> ATB
>> Dougie.
>> P.S. Have you ever read any of Le Carre's novels?  He has written 23 and
>> like all good security and intelligence buffs, I have read, nay, studied
>> all of them.  You can find them in all good bookshops and they are
>> available on line very cheap.  Like le Carre, I find fiction far more
>> educational than non-fiction.  And fiction has the added advantage that it
>> is not classified or subject to secrecy laws, yet contains all the craft of
>> the trade of information.  Beware of course, that le Carre's main theme, as
>> is that of William Boyd, is betrayal...
>> On 26/06/14 19:54, Aftermath wrote:
>> ... cant tell if sarcasm or not..?
>>  but yeah, I just found the link and I thought this list would
>> appreciate the share :)
>>  On Thu, Jun 26, 2014 at 11:48 AM, doug <douglasrankine2001@xxxxxxxxxxx>
>> wrote:
>>> On 26/06/14 18:19, Aftermath wrote:
>>> Dear Aftermath...or is it Bill?
>>> I thought this posting was absolutely brilliant. My congratulations to
>>> you for bringing it to our attention and Mr. Wong wh did the initial
>>> research.  Why?  Because...if you think on it, you have saved the NSA, the
>>> CIA and the intelligence and security departments  of the New York Police
>>> Force huge amounts of money on expenditure of resources...and foreign
>>> organisations such as the Chinese, the Israelis, the Russians and any ole
>>> Tom, Dick and Harry of a security organisation a lot of time.  Even the
>>> United Nations security and intelligence services will be all agog at your
>>> posting when they read it.  I hope you are not claiming copyright on
>>> it...;-) . At the click of a button, they can all find out  who has been
>>> doing what in New York....United Nations officials, Ambassadors, envoys,
>>> consuls and all sorts of public and civil service officials, secret
>>> service, intelligence services, even the private sector.  All this metadata
>>> available to link up with known associations and links...and all open
>>> source too, who visited who and when, who was where and when, with a little
>>> help from the smart mobile phone, the most valuable source of i.d. location
>>> and contacts, you have provided the world, with a wealth of information.
>>>  Just think...the amount of money you have saved the world's intelligence
>>> organisations.  Absolutely f*cking brilliant....My congratulations...
>>> ATB;-)
>>> Dougie.
>>> P.S. Who needs privacy and encryption when we have all this stuff about
>>> the world leaders in our very own hands.  It is all a question of pressing
>>> the right button...
>>>  http://chriswhong.com/open-data/foil_nyc_taxi/
>>>> and
>>>> https://medium.com/@vijayp/f6bc289679a1
>>>> from the second link...
>>>> "
>>>> Recently, thanks to a Freedom of Information request, Chris
>>>> Whongreceived and made public a complete dump of historical trip and fare
>>>> logs from NYC taxis. It’s pretty incredible: there are over 20GB of
>>>> uncompressed data comprising more than 173 million individual trips. Each
>>>> trip record includes the pickup and dropoff location and time, anonymized
>>>> hack licence number and medallion number (i.e. the taxi’s unique id number,
>>>> 3F38, in my photo above), and other metadata.
>>>> These data are a veritable trove for people who love cities, transit,
>>>> and data visualization. But there’s a big problem: the personally
>>>> identifiable information (the driver’s licence number and taxi number)
>>>> hasn’t been anonymized properly — what’s worse, it’s trivial to undo, and
>>>> with other publicly available data, one can even figure out which person
>>>> drove each trip. In the rest of this post, I’ll describe the structure of
>>>> the data, what the person/people who released the data did wrong, how easy
>>>> it is to deanonymize, and the lessons other agencies should learn from
>>>> this. (And yes, I’ll also explain how rainbows fit in).
>>>> The NYC taxi data consist of a number of CSV-files.....
>>>> "

Other related posts: