[cryptome] Re: NYC Taxicab Log Dump
- From: Andrew Hornback <achornback@xxxxxxxxx>
- To: cryptome@xxxxxxxxxxxxx
- Date: Thu, 26 Jun 2014 17:50:00 -0400
I've read Bamford's Shadow Factory... recommended.
--- A
On Thu, Jun 26, 2014 at 5:42 PM, Aftermath <aftermath.thegreat@xxxxxxxxx>
wrote:
> I will look into those..
>
> I have not read Puzzle Place, but I read Body of Secrets by the same
> author. Deeply fascinating
>
>
> http://en.wikipedia.org/wiki/Body_of_Secrets:_Anatomy_of_the_Ultra-Secret_National_Security_Agency
>
>
>
> On Thu, Jun 26, 2014 at 1:30 PM, doug <douglasrankine2001@xxxxxxxxxxx>
> wrote:
>
>> Excellent...I am very pleased to have added to your depth and breadth
>> of education...as you have added to mine. And I am very pleased that we
>> hold at least one author in common. I am always grateful when someone adds
>> to my education or makes me think or act in a new way.
>>
>> Can I also suggest a book by Norman Mailer...a bit dated now, and of
>> course he died the other year without writing the sequel. It is called
>> "Harlot's Ghost", available at all good bookshops...and a used edition is
>> very cheap at Amazon...if I am allowed to advertise such things here on the
>> list...though one can also get it on hire from ones local library for
>> free...if there is such an organisation these days.
>>
>> In the book you will discover a number of things about the CIA and such
>> like. How the organisation is run, the compartmentalisation and how it
>> works for and against the efficiency of the organisation and how it
>> protects those who don't want to be brought out into the heat, the names of
>> the families who were involved in its creation, their ancestors and
>> descendants who are still involved (pseudonyms of course) and such like.
>> Mailer spent many years researching it, and it is a bit dated now, but the
>> essence is still there and it is a fascinating story. Puzzle Palace was my
>> first introduction to the intelligence game and after reading it, I was
>> hooked.
>>
>> Also, there is something in there which helps all good spies...or dare I
>> say, "spy buffs" in the book to memorise things like phone numbers and
>> various other kinds of numbers, data and personal information and such
>> stuff. None of this stuff is new of course, neither is it classified, and
>> one can find a plethora of websites on line which offer various methods of
>> improving one's memory, though one should never make the mistake that such
>> improvement attaches to ones intellect and also, like many aspects of
>> cryptography and security, there is a lot of snake oil around, which costs
>> a lot of money. I am one of those who believes that mind mapping and the
>> use of memory techniques can save one a lot of time, as well as helping
>> with assimilation and retention of information; but I have never been able
>> to make up my mind whether they in fact do so, or are just a trick to get
>> one to spend some money. I am a great believer that knowledge should be
>> free and open to everyone who cares to educate themselves.
>>
>> Mind you the Ancient Greeks and Romans used such methods, and there are
>> a number of biographies which suggest that great leaders used such skills
>> to remember things like battle plans or complex algorithms or even the
>> names of every soldier in their armies, such as Alexander...however, he
>> died early, so such knowledge didn't save him.
>>
>>
>> On 26/06/14 21:06, Aftermath wrote:
>>
>> OK :) and no, Ill have to look into Le Carre's novels.
>>
>> And I prefer William ;)
>>
>>
>>
>>
>> On Thu, Jun 26, 2014 at 12:31 PM, doug <douglasrankine2001@xxxxxxxxxxx>
>> wrote:
>>
>>> Dear Aftermath...or is it Bill,
>>>
>>> One of the things that I learned early in life, particularly when I
>>> moved to London and then to the Isle of Wight, was that taxi drivers are a
>>> mine of information on the local environment. When one is contemplating
>>> moving into a new area, then what can be better than entering into
>>> conversation with one's taxi driver to find out who is who and who does
>>> what in the local environment. Of course, one doesn't have to enter into
>>> conversation with one's cabbie, after all, one is a captive audience, one
>>> just has to guide the conversation with a few "open" questions which helps
>>> to ascertain the kind of environment and type of people one is going or
>>> hoping to meet up with if one settles there. The nice thing about such
>>> open source messages and conversations is that they, up until now, is that
>>> they were private and not recorded; and can establish so much of what one
>>> wants to know, without any additional expense rather than the hire. Of
>>> course, nowadays, the metadata is just as important...(if not more
>>> so)...than the content of the conversation, but that is the internet and
>>> all this new information technology for you.
>>>
>>> When I visited New York for a holiday...before 9/11, I met up with a
>>> cabbie whose English was very poor. He was a Russian immigrant, yet knew
>>> his way around New York so well, and was able to tell me and me missus the
>>> best places to go and which places to avoid and at what time. All it cost
>>> was a nice tip at the end of the journey, with which he was very pleased,
>>> and he could see that us Scots were not at all prejudiced about his origins
>>> or his politics. What more can one ask for in life...sarcasm is such an
>>> ugly word...to be accused of using it is so hurtful... ;-) .
>>> ATB
>>> Dougie.
>>> P.S. Have you ever read any of Le Carre's novels? He has written 23 and
>>> like all good security and intelligence buffs, I have read, nay, studied
>>> all of them. You can find them in all good bookshops and they are
>>> available on line very cheap. Like le Carre, I find fiction far more
>>> educational than non-fiction. And fiction has the added advantage that it
>>> is not classified or subject to secrecy laws, yet contains all the craft of
>>> the trade of information. Beware of course, that le Carre's main theme, as
>>> is that of William Boyd, is betrayal...
>>>
>>>
>>> On 26/06/14 19:54, Aftermath wrote:
>>>
>>> ... cant tell if sarcasm or not..?
>>>
>>>
>>> but yeah, I just found the link and I thought this list would
>>> appreciate the share :)
>>>
>>>
>>> On Thu, Jun 26, 2014 at 11:48 AM, doug <douglasrankine2001@xxxxxxxxxxx>
>>> wrote:
>>>
>>>> On 26/06/14 18:19, Aftermath wrote:
>>>> Dear Aftermath...or is it Bill?
>>>>
>>>> I thought this posting was absolutely brilliant. My congratulations to
>>>> you for bringing it to our attention and Mr. Wong wh did the initial
>>>> research. Why? Because...if you think on it, you have saved the NSA, the
>>>> CIA and the intelligence and security departments of the New York Police
>>>> Force huge amounts of money on expenditure of resources...and foreign
>>>> organisations such as the Chinese, the Israelis, the Russians and any ole
>>>> Tom, Dick and Harry of a security organisation a lot of time. Even the
>>>> United Nations security and intelligence services will be all agog at your
>>>> posting when they read it. I hope you are not claiming copyright on
>>>> it...;-) . At the click of a button, they can all find out who has been
>>>> doing what in New York....United Nations officials, Ambassadors, envoys,
>>>> consuls and all sorts of public and civil service officials, secret
>>>> service, intelligence services, even the private sector. All this metadata
>>>> available to link up with known associations and links...and all open
>>>> source too, who visited who and when, who was where and when, with a little
>>>> help from the smart mobile phone, the most valuable source of i.d. location
>>>> and contacts, you have provided the world, with a wealth of information.
>>>> Just think...the amount of money you have saved the world's intelligence
>>>> organisations. Absolutely f*cking brilliant....My congratulations...
>>>> ATB;-)
>>>> Dougie.
>>>> P.S. Who needs privacy and encryption when we have all this stuff about
>>>> the world leaders in our very own hands. It is all a question of pressing
>>>> the right button...
>>>>
>>>>
>>>> http://chriswhong.com/open-data/foil_nyc_taxi/
>>>>> and
>>>>> https://medium.com/@vijayp/f6bc289679a1
>>>>>
>>>>>
>>>>> from the second link...
>>>>>
>>>>> "
>>>>> Recently, thanks to a Freedom of Information request, Chris
>>>>> Whongreceived and made public a complete dump of historical trip and fare
>>>>> logs from NYC taxis. It’s pretty incredible: there are over 20GB of
>>>>> uncompressed data comprising more than 173 million individual trips. Each
>>>>> trip record includes the pickup and dropoff location and time, anonymized
>>>>> hack licence number and medallion number (i.e. the taxi’s unique id
>>>>> number,
>>>>> 3F38, in my photo above), and other metadata.
>>>>>
>>>>> These data are a veritable trove for people who love cities, transit,
>>>>> and data visualization. But there’s a big problem: the personally
>>>>> identifiable information (the driver’s licence number and taxi number)
>>>>> hasn’t been anonymized properly — what’s worse, it’s trivial to undo, and
>>>>> with other publicly available data, one can even figure out which person
>>>>> drove each trip. In the rest of this post, I’ll describe the structure of
>>>>> the data, what the person/people who released the data did wrong, how easy
>>>>> it is to deanonymize, and the lessons other agencies should learn from
>>>>> this. (And yes, I’ll also explain how rainbows fit in).
>>>>>
>>>>>
>>>>> The NYC taxi data consist of a number of CSV-files.....
>>>>> "
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
Other related posts:
