[cryptome] Re: NYC Taxicab Log Dump
- From: Shaun O'Connor <capricorn8159@xxxxxxxxx>
- To: cryptome@xxxxxxxxxxxxx
- Date: Thu, 26 Jun 2014 22:37:45 +0100
Miles copeland wrote some informative books relating to the intelligence
community as well.and it was his work that stimulated my interest one
book in particular( the tittle of which eludes me) gave an interesting
methodology for creating otp
On 26/06/2014 21:30, doug wrote:
> Excellent...I am very pleased to have added to your depth and breadth
> of education...as you have added to mine. And I am very pleased that
> we hold at least one author in common. I am always grateful when
> someone adds to my education or makes me think or act in a new way.
>
> Can I also suggest a book by Norman Mailer...a bit dated now, and of
> course he died the other year without writing the sequel. It is
> called "Harlot's Ghost", available at all good bookshops...and a used
> edition is very cheap at Amazon...if I am allowed to advertise such
> things here on the list...though one can also get it on hire from ones
> local library for free...if there is such an organisation these days.
>
> In the book you will discover a number of things about the CIA and
> such like. How the organisation is run, the compartmentalisation and
> how it works for and against the efficiency of the organisation and
> how it protects those who don't want to be brought out into the heat,
> the names of the families who were involved in its creation, their
> ancestors and descendants who are still involved (pseudonyms of
> course) and such like. Mailer spent many years researching it, and it
> is a bit dated now, but the essence is still there and it is a
> fascinating story. Puzzle Palace was my first introduction to the
> intelligence game and after reading it, I was hooked.
>
> Also, there is something in there which helps all good spies...or dare
> I say, "spy buffs" in the book to memorise things like phone numbers
> and various other kinds of numbers, data and personal information and
> such stuff. None of this stuff is new of course, neither is it
> classified, and one can find a plethora of websites on line which
> offer various methods of improving one's memory, though one should
> never make the mistake that such improvement attaches to ones
> intellect and also, like many aspects of cryptography and security,
> there is a lot of snake oil around, which costs a lot of money. I am
> one of those who believes that mind mapping and the use of memory
> techniques can save one a lot of time, as well as helping with
> assimilation and retention of information; but I have never been able
> to make up my mind whether they in fact do so, or are just a trick to
> get one to spend some money. I am a great believer that knowledge
> should be free and open to everyone who cares to educate themselves.
>
> Mind you the Ancient Greeks and Romans used such methods, and there
> are a number of biographies which suggest that great leaders used such
> skills to remember things like battle plans or complex algorithms or
> even the names of every soldier in their armies, such as
> Alexander...however, he died early, so such knowledge didn't save him.
>
> On 26/06/14 21:06, Aftermath wrote:
>> OK :) and no, Ill have to look into Le Carre's novels.
>>
>> And I prefer William ;)
>>
>>
>>
>>
>> On Thu, Jun 26, 2014 at 12:31 PM, doug
>> <douglasrankine2001@xxxxxxxxxxx
>> <mailto:douglasrankine2001@xxxxxxxxxxx>> wrote:
>>
>> Dear Aftermath...or is it Bill,
>>
>> One of the things that I learned early in life, particularly when
>> I moved to London and then to the Isle of Wight, was that taxi
>> drivers are a mine of information on the local environment. When
>> one is contemplating moving into a new area, then what can be
>> better than entering into conversation with one's taxi driver to
>> find out who is who and who does what in the local environment.
>> Of course, one doesn't have to enter into conversation with one's
>> cabbie, after all, one is a captive audience, one just has to
>> guide the conversation with a few "open" questions which helps to
>> ascertain the kind of environment and type of people one is going
>> or hoping to meet up with if one settles there. The nice thing
>> about such open source messages and conversations is that they,
>> up until now, is that they were private and not recorded; and can
>> establish so much of what one wants to know, without any
>> additional expense rather than the hire. Of course, nowadays, the
>> metadata is just as important...(if not more so)...than the
>> content of the conversation, but that is the internet and all
>> this new information technology for you.
>>
>> When I visited New York for a holiday...before 9/11, I met up
>> with a cabbie whose English was very poor. He was a Russian
>> immigrant, yet knew his way around New York so well, and was able
>> to tell me and me missus the best places to go and which places
>> to avoid and at what time. All it cost was a nice tip at the end
>> of the journey, with which he was very pleased, and he could see
>> that us Scots were not at all prejudiced about his origins or his
>> politics. What more can one ask for in life...sarcasm is such an
>> ugly word...to be accused of using it is so hurtful...;-) .
>> ATB
>> Dougie.
>> P.S. Have you ever read any of Le Carre's novels? He has written
>> 23 and like all good security and intelligence buffs, I have
>> read, nay, studied all of them. You can find them in all good
>> bookshops and they are available on line very cheap. Like le
>> Carre, I find fiction far more educational than non-fiction. And
>> fiction has the added advantage that it is not classified or
>> subject to secrecy laws, yet contains all the craft of the trade
>> of information. Beware of course, that le Carre's main theme, as
>> is that of William Boyd, is betrayal...
>>
>>
>> On 26/06/14 19:54, Aftermath wrote:
>>> ... cant tell if sarcasm or not..?
>>>
>>>
>>> but yeah, I just found the link and I thought this list would
>>> appreciate the share :)
>>>
>>>
>>> On Thu, Jun 26, 2014 at 11:48 AM, doug
>>> <douglasrankine2001@xxxxxxxxxxx
>>> <mailto:douglasrankine2001@xxxxxxxxxxx>> wrote:
>>>
>>> On 26/06/14 18:19, Aftermath wrote:
>>> Dear Aftermath...or is it Bill?
>>>
>>> I thought this posting was absolutely brilliant. My
>>> congratulations to you for bringing it to our attention and
>>> Mr. Wong wh did the initial research. Why? Because...if
>>> you think on it, you have saved the NSA, the CIA and the
>>> intelligence and security departments of the New York
>>> Police Force huge amounts of money on expenditure of
>>> resources...and foreign organisations such as the Chinese,
>>> the Israelis, the Russians and any ole Tom, Dick and Harry
>>> of a security organisation a lot of time. Even the United
>>> Nations security and intelligence services will be all agog
>>> at your posting when they read it. I hope you are not
>>> claiming copyright on it...;-) . At the click of a button,
>>> they can all find out who has been doing what in New
>>> York....United Nations officials, Ambassadors, envoys,
>>> consuls and all sorts of public and civil service officials,
>>> secret service, intelligence services, even the private
>>> sector. All this metadata available to link up with known
>>> associations and links...and all open source too, who
>>> visited who and when, who was where and when, with a little
>>> help from the smart mobile phone, the most valuable source
>>> of i.d. location and contacts, you have provided the world,
>>> with a wealth of information. Just think...the amount of
>>> money you have saved the world's intelligence organisations.
>>> Absolutely f*cking brilliant....My congratulations...
>>> ATB;-)
>>> Dougie.
>>> P.S. Who needs privacy and encryption when we have all this
>>> stuff about the world leaders in our very own hands. It is
>>> all a question of pressing the right button...
>>>
>>>
>>> http://chriswhong.com/open-data/foil_nyc_taxi/
>>> and
>>> https://medium.com/@vijayp/f6bc289679a1
>>>
>>>
>>> from the second link...
>>>
>>> "
>>> Recently, thanks to a Freedom of Information request,
>>> Chris Whongreceived and made public a complete dump of
>>> historical trip and fare logs from NYC taxis. It’s
>>> pretty incredible: there are over 20GB of uncompressed
>>> data comprising more than 173 million individual trips.
>>> Each trip record includes the pickup and dropoff
>>> location and time, anonymized hack licence number and
>>> medallion number (i.e. the taxi’s unique id number,
>>> 3F38, in my photo above), and other metadata.
>>>
>>> These data are a veritable trove for people who love
>>> cities, transit, and data visualization. But there’s a
>>> big problem: the personally identifiable information
>>> (the driver’s licence number and taxi number) hasn’t
>>> been anonymized properly — what’s worse, it’s trivial to
>>> undo, and with other publicly available data, one can
>>> even figure out which person drove each trip. In the
>>> rest of this post, I’ll describe the structure of the
>>> data, what the person/people who released the data did
>>> wrong, how easy it is to deanonymize, and the lessons
>>> other agencies should learn from this. (And yes, I’ll
>>> also explain how rainbows fit in).
>>>
>>>
>>> The NYC taxi data consist of a number of CSV-files.....
>>> "
>>>
>>>
>>>
>>>
>>
>>
>
--
*_PRIVACY IS A BASIC RIGHT - NOT A CONCESSION _*
Other related posts:
