November 2014 - CollEc-run

Re: [CollEc] CollEc
by Thomas Krichel 21 Nov '14

21 Nov '14

Nikos Askitas writes > I hope that you remember me I have a very poor memory, but I remember. > I have been looking at CollEc recently and started programming the > search interface a bit. CollEc has a search interface. > I would basically like to write a script which will get the shortest > path(s) between two authors. So I would like to say > get_paths(Askitas, Krichel) and get the data in some form I can work > on further. I can strip the HTML with regexs I'd rather give you the complete dataset. > 1. Is there a file which maps RealName to repec handles? Can you point me somewhere? icanis@katri:~/icanis/input$ cat ras_mans_nodes_1416097129.xml | head -10 <nodes> <node ref="paa1" name="Arild Aakvik" homepage="http://people.uib.no/secaa"/> <node ref="paa11" name="Arnstein Aassve" homepage="http://faculty.unibocconi.eu/arnsteinaassve/"/> <node ref="paa12" name="susan aaronson" homepage="http://www.gwu.edu/~elliott/faculty/aaronson.cfm"/> <node ref="paa13" name="Daniel Aaronson"/> <node ref="paa2" name="David Aadland" homepage="http://www.uwyo.edu/aadland/"/> <node ref="paa22" name="Knut Are Aastveit" homepage="http://www.norges-bank.no/en/about/research/economists/aastveit-knut-are/"/> <node ref="paa23" name="Knut Aase" homepage="http://www.nhh.no/Default.aspx?ID=2004"/> <node ref="paa6" name="Rolf Aaberge"/> <node ref="paa8" name="Rob Aalbers"/> > 2. Do you have a script somewhere which would work for pairs of names? No, but I have the data. Let's look at at example icanis@katri:~/icanis/input$ grep Krichel ras_mans_nodes_1416097129.xml <node ref="pkr1" name="Thomas Krichel" homepage="http://openlib.org/home/krichel"/> icanis@katri:~/icanis/input$ grep Askitas ras_mans_nodes_1416097129.xml <node ref="pas55" name="Nikos Askitas" homepage="http://www.iza.org/home/askitas"/> This gives us the handle, from there we can find the paths in two ways. icanis@katri:~$ grep 'pas55$' icanis/paths/ras/biwe/k/r/1/paths 5 pzi1 pur18 psc166 pzi13 pas55 icanis@katri:~$ grep 'pkr1$' icanis/paths/ras/biwe/a/s/55/paths 5 pzi13 psc166 pur18 pzi1 pkr1 I use the latest of the two path files that I can grep. That's how the search works. BTW, there are close to 1 billion paths. > The search cgi you have now works for Name and handle or for two > handles. Yes, I should write an interface for two. > 3. Is there a cgi you can post which will deliver shortest paths as > arrays of arrays without html? Even a path per line separated by > comma would do. I have not done this, but you could do it. > 4. Would you share ALL the coauthorship data with me? Sure. Currently I don't have a backup of the paths, since they can be recomputed. But it would take a very long time. If you would provide a backup for me that would help me. Kindly open an account "icanis" on a machine with rsync installed and authorise the key icanis@katri:~$ cat ~/.ssh/id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOhbsMyNTv9O3h0tu7eMjqdR5t2BG1jxqErz2YVoVcPGfSZ2mOgwMLMSeFQY3YwNzkC/lfzbtGKybm9dl+CfvnERKL3tA5+NiQiXyJ0WZVp/bEvePMMsDzcLv+Z4RHStl9mU+9OuTPDYzpheTWbjswszQc9kE2ECqlRUqfIb1MQ1/UWDfaq9p8O0USYc25t/uM95VOtHTPHvy75d2+8m3NJPhdFX2yqTiD0ra2phdwMrgw+Csivgs/gfhIhMepep29+V+sBvHDhuVOVBS9N87ae7AJlw5HDceIKwkxasFfMB9mw7RxK7OGTGBcM78ykFRustlKAiDy3M9t/iqb9NNh icanis@fricka This is the size of the account right now icanis@katri:~$ du -s . 81919236 . > I would like to write some papers with it? Write as many or as few as you like. Three additional points. (1) If you can calculate summary statistics on the network, I can use them in the web site. The site is written using XSLT, so an XML file with the stats will be perfect. (2) I have taken liberty to add you to the Collec-run mailing list. You are the sixth member on the list. It has very little traffic, but all correspondance about CollEc should be sent there. (3) I was in Cologne at GESIS between October 13 and 31. I hiked even do the Rheinsteig between Bonn and the Drachenfels on 25th October so I could have met you. But I had forgotten about you, sorry. I am back in NYC now, homeless and jobless, so if you have a consultancy opportunity I would look into it. I will probably be back at GESIS at some stage. -- Cheers, Thomas Krichel http://openlib.org/home/krichel skype:thomaskrichel

1 0

Re: [CollEc] repec genealogy/citation/coauthor raw data
by Lars Vilhuber 08 Nov '14

08 Nov '14

P. S. What lead you to pick (single or sometimes dual) advisor, and not allowing for committee members? I've heard it a few times from folks I showed the genealogy page. Any plans to add that capability? -- Sent from my mobile device On Nov 7, 2014 2:02 PM, 'Christian Zimmermann' <zimmermann(a)stlouisfed.org> wrote: Hallo lars, gaebe es die Tage einen guten Zeitpunkt zu telefonieren? Eine ganz praktische Frage. Wir finden in unseren OH QCEW daten relativ viele Firmen, die nur einen quarter existieren (~20%). Denkst du das ist eine realistische Groessenordnung? Viele Gruesse, Till

2 1

Re: [CollEc] repec genealogy/citation/coauthor raw data
by Lars Vilhuber 07 Nov '14

07 Nov '14

Then send me a mysqldump. That'll do just fine. -- Sent from my mobile device On Nov 7, 2014 2:02 PM, 'Christian Zimmermann' <zimmermann(a)stlouisfed.org> wrote: Tell me in what format you want the genealogy data. I have it in MySQL. Christian Zimmermann FIGUGEGL! Economic Research Federal Reserve Bank of St. Louis P.O. Box 442 St. Louis MO 63166-0442 USA http://ideas.repec.org/zimm/ @CZimm_economist On Fri, 7 Nov 2014, Lars Vilhuber wrote: > Christian, Thomas: > > just a brief update. We've identified a (very) qualified (undergrad) RA, who'll work on this part time starting mid-January. > > We'll start with a demo using a (static) data dump (I have grabbed a one-time copy of Thomas' data, will still need a dump of the genealogy data, unless the RA simply decides to scrape it), we can discuss how to most efficiently "collect" the data necessary for graphing and exploring in an "implemented production" server, once we get there. > > I'm excited! > > Lars > > -- > Lars Vilhuber | Executive Director > Economist | Labor Dynamics Institute > lars.vilhuber(a)cornell.edu -+- Cornell University > Office/Cell : +1.607-330-5743 | ILR School - Department of Economics > Fax : +1.866-873-9078 + U.S. Census Bureau - CES - LEHD > > Labor Dynamics Institute: http://www.ilr.cornell.edu/ldi > Visit the Cornell VirtualRDC: http://www.vrdc.cornell.edu > > ________________________________________ > From: 'Christian Zimmermann' <zimmermann(a)stlouisfed.org> > Sent: Tuesday, October 28, 2014 11:03 > To: Lars Vilhuber > Cc: Thomas Krichel; CollEc Run > Subject: Re: repec genealogy/citation/coauthor raw data > > I do not (yet) have a way to share the geneaology data, but we can talk > about what would best suitable. > > My optimal outcome would to have an app that can be used on CollEc, your > site and the RePEc Genealogy to show the respective networks. CollEc and > RG would be limited to their data, you could show all you want. > > Christian Zimmermann FIGUGEGL! > Economic Research > Federal Reserve Bank of St. Louis > P.O. Box 442 > St. Louis MO 63166-0442 USA > http://ideas.repec.org/zimm/ @CZimm_economist > > On Tue, 28 Oct 2014, Lars Vilhuber wrote: > >> Thomas, Christian: >> >> (I'll continue the discussion in English, since I might want to forward this to my RAs) >> >> This is doable! >> >> I may have a very competent undergraduate RA to work on this in the Spring. I'll have more details (on his availability) in about 2 weeks. >> >> I'm putting together a quick write-up of why this makes sense from our projects, and why it might make sense for RePEc (or in general). I'm aiming for something that would be maintainable as an actual web app on (your? our?) servers as part of the network. >> >> I can see all the data (I think) I need on the page that Christian sent me, except for the genealogy data. Possibly Thomas has data in a more concise format than us parsing the RePEc:per archive? But that's no problem. Can you provide me with a one-time dump (extract probably is NOT sufficient if done randomly, but could be time-based - birthday of the person in the database?) for now of the genealogy database? (what format are you using there? ) >> -- >> Lars Vilhuber | Executive Director >> Economist | Labor Dynamics Institute >> lars.vilhuber(a)cornell.edu -+- Cornell University >> Office/Cell : +1.607-330-5743 | ILR School - Department of Economics >> Fax : +1.866-873-9078 + U.S. Census Bureau - CES - LEHD >> >> Labor Dynamics Institute: http://www.ilr.cornell.edu/ldi >> Visit the Cornell VirtualRDC: http://www.vrdc.cornell.edu >> >> ________________________________________ >> From: Thomas Krichel <krichel(a)openlib.org> >> Sent: Saturday, October 25, 2014 03:17 >> To: Lars Vilhuber >> Cc: CollEc Run; Christian Zimmermann >> Subject: Re: repec genealogy/citation/coauthor raw data >> >> Lars Vilhuber wrote: >> >>> this is what I was aiming to do: >>> >>> http://www.vrdc.cornell.edu/repecgraph/ >>> >>> in my abundant free time. I have a programmer who would be perfectly >>> capable of doing that, but no time right now (may change in a >>> year). Also, if you are interested, I could propose this to one of >>> our Cornell CS classes - they do 'client-oriented visualization >>> projects' and could probably do this with a dump of the data, if not >>> an API. >>> >>> This is just idle thoughts, but who knows... >> >> 'Christian Zimmermann' writes >> >>> I always wanted http://collec.repec.org/ to have something like this. >> >> Me to. But I don't have the expertise. >> >>> Thomas may help you getting access to the CollEc data/server for >>> starters. >> >> I surely will be pleased to do that. >> >> >> -- >> >> Cheers, >> >> Thomas Krichel http://openlib.org/home/krichel >> skype:thomaskrichel >> >

1 0

Re: [CollEc] repec genealogy/citation/coauthor raw data
by Thomas Krichel 07 Nov '14

07 Nov '14

Lars Vilhuber wrote: > this is what I was aiming to do: > > http://www.vrdc.cornell.edu/repecgraph/ > > in my abundant free time. I have a programmer who would be perfectly > capable of doing that, but no time right now (may change in a > year). Also, if you are interested, I could propose this to one of > our Cornell CS classes - they do 'client-oriented visualization > projects' and could probably do this with a dump of the data, if not > an API. > > This is just idle thoughts, but who knows... 'Christian Zimmermann' writes > I always wanted http://collec.repec.org/ to have something like this. Me to. But I don't have the expertise. > Thomas may help you getting access to the CollEc data/server for > starters. I surely will be pleased to do that. -- Cheers, Thomas Krichel http://openlib.org/home/krichel skype:thomaskrichel

3 7