Lets get to work. I want to rewrite the intro to my muniporject (again) the basic idea is that we need to first account for the urban area progress before we can say anything about the incentives of the municipalities that make up the urban area. I want to relate this all to the Oblivious equilbrium literature, where actions are taken with the long run stationary EQ in mind. A merger would be a way to alter this long run EQ.
Lets Call it a day. I think theres something here. I beleive that I have now split my dissertation into three chapters
Deriving Firms Expectations Over Equilbrium Urban Area Structure
Market Failures in Municipalities
Estimating Subcenters
1 and 2 relate directly to municipal mergers. The first paper, likely my job market paper (because it has fancy math) lets us find the effect of municipal market structure on urban area preformance. The second paper gives reasons why the municipal market would likely have market failures while higher levels of competititon would not. The third is techinical, but its just McMillen 2001 with modern tools.
What to do for DM
Throw more citiations in to the README chernuzukov for testing
its not non-smooth, (around where we switch moments)
output distribution of Tlist
preform, not perform
more bolier plate “test results at dm related parameter
test results below parameter
brackets around confidence sets
make a new dataset
say something about relative scale of parameters
max function is only for esitmtion and testing. Search through with MMM
To find confidence set, use TWO STEP
To test a particular point, use Max Functino
NEed a methdo in STATA to test a particular point
Point identified - use max for everything
IF we are below zero, switch form max to MMM
report identified set and confidence set
find identified set using search procedure
nice to know something about size of expectational error vs idoyncratic error
problem is in fixed effects
Optimizer is not working.
use MCMC with likelihood
only use max function for testing.
What did we do to get it working?
This ones for all the The Mamas & The Papas fans. Lets work on the FKRB project today. What do we want to get done?
Finally have my head above water, and I am going to spend the rest of the night restructing the site. Lets made a to-do list, and try not to go off on tangents. The major change that I am going to make is that I am not going to have a public mini-blog. That seems like a bit too much information. The blog is still going to be up, and Im going to use it a bit more throughout the day, but the miniblogs were getting cumbersome, and I’m not sure if basically just writting out text verbatim from the text is all that legal/ethical when I am doing paper summaries.
TODO
OK. Let us go.
Do I need an about me page? I think I say everything that I want to say about myself on the homepage. Or I can at least add a couple things so that I can remove it. Less is more sometimes
Im done today and will push. I think I have the intro to the muniproject finsihed tho
I again forgot to push this out. Ill do it today I swear. I am trying to finish the DM project. I have the readme in a place that is pretty good, just need the code to be a touch nicer. It runs, but its incredibly ugly. Instead of trying to rebuild Rome in a day, lets work on the worst parts of the code and get those better. As we make gradual improvements, we will eventually get to a place where the code is not as bad.
We need to do two things, get the options to line up with the readme file. Theres no reason that the user should have to use our jargon, lets make it easy on them. We also need to make sure the default are actually the defaults.
After, I need to make the testing methods static. creating a new object every test is wrecking memory and causign it to be incredibly slow.
Lets get those two things done. After that, we can very easily finish the readme.
What needs to change
Done, done and done with those checks. The notation for
I realized that I did not push any of my work yesterday. All good, its will just get put out with todays.
I am going to focus on finsihed the updated readme for the DM project today, and hopefully start on the simute_dmreg.ado and neccessary java modifications to get that done.
Lets go!
I realized that our estimator is not ready to go out of the box. I dont have anything to convert user data into a form that is acceptable for the Andres and Shi (2014) procedure. I think there might be more things like this, so I’m going to create a running list
Were doing 4 things today:
Lets go!
Its a new week and lets make it more productive than the last. The previous week was interesting as it was the first full work week where I worked exclusively on this site. I really like the structure that the site provides and I think that it is making me a bit more efficient. That being said, after a week of doing this I think theres a change that I would like to make. I will probably instate this change tonight, as it will require a bit of site restructing.
I think that the categorization of everything is not quite where it
should be. I think the blog should be a stand alone section that
contains basic thoughts and daily agendas. Further, I don’t think I want
to have as big of a separation between paper summaries and projects. So
the new format is going to have the blog on its own, and then just a
singular Projects section. Within the project section, I will have
several sections (think headers) all on the same page with links to the
various topics. So the projects page will have a header called Works in
Progress for things that are not finished. Once something is finished,
it will be catagorize into either Paper Summaries, Papers, or
Memorandums. An example of each category will probably help. My McMillen
2001 paper summary would go into the paper summaries section. The KLR
2023 will still be in the works in progress, but will eventually move
into the paper summaries section. My three research projects are works
in progress, but will eventually be moved into a section called Papers
(when that change is made I’m not sure, probably when it goes out into
the world). All other topics like the FFF (that I missed) and the deep
dive into the mvrnorm function will eventually find a home
in the Memorandums section.
So thats the change. But for the actual work day, I want to finish
the mvrnorm memo, then I should work on KLR 2023. After
working on that, I think I want to refresh the intro to the MuniProject,
including a ``big picture” section that I discussed with Bart.
So thats the plan. Lets go.
Just had a meeting with my advisor. Seems interested in the FKRB stuff, but wants to talk more about our Dickstein Morales project. I’ll need to fit that into my schedule. As I am an RA on that project, I’m not sure I want to share all the details of that on here, so its mostly going to be offline work.
Heres what i need to do for our next meeting
So theres that. I got to a good point on the mvrnorm
memo, so I am going to switch gears and start back up on the KLR 2023
paper summary
I want to simulate municipal mergers. Do I need the entire US, or can I just focus on St. Louis?
Okay. Its Fast Function Friday. Should that be here, or should i link to it? I’m going to be doing the FKRB stuff and optimizing that, so its the work is going to take place in the mini-blog and replication. But where does it go after? We will find out. The schedule for today is
Lets go.
Lets make today productive. We need to do three things - Lets implement McMillen 2001 in the MuniProject replication page - Lets get everything from the old FKRB project onto the website - Lets get the intial reading of the KLR 2023 paper into the mini blog
Accomplishing those goals constitutes a productive day. Lets go.
Wow, I did not write much here at all. Check out the MuniProject page for all of todays work. I was able to fix the myRender function a bit, and I was able to get all the data done in the MuniProject. Disney world and airports (and just random missing zipcodes) really made today and issue. But back at it tomrrow/later tonight after tennis
I had three things on my to do list yesterday and I almost finished 1 1/2. As a quick recap, I am almost done with the McMillen 2001 paper summary. I just need to finish the LWR part that is in the appendix. The census scrape is doing something weird after 2004, I cant get anything to run after that year. I’ll have to figure that out. Finally, I want to get some work down on FKRB.
Before laying out my schedule, Ill mention that I’m trying to get a Linux distro on my home laptop. In my struggles, I got weak and went with a pretty large distro, Linux Mint: XFCE. The install didn’t go great, something about the wifi drivers. So to make progress there, I’m going to go back to my guiding, No Black Boxes, principle and try to install tinyLinux CorePlus. I know, I should be using the Micro, but baby steps.
With that out of the way, lets talk schedule.
The last thing is for the absolute end of the day so I can work on it at home. If I get the first two done, I would also like to figure out whats going on with the rest of the scrape.
Ok. Lets go
Yesterday I ran into a bunch of issue with rendering. I should probably figure that out. I’m thinking thats a weekend project though.
I just finished the draft of the McMillen Paper summary. Should I edit now? Probably, but fresh eyes will help right? Onto stuff for the reading group.
I was just working on the muniproject and I had a good idea. Theres a lot of things that I want to do at the end of the day, just because its going to take a lot of computational time, and I can just let it run overnight. I should create a section at the end of each days post that collects that. At the end of the day, I will do those, and write a little sign off, and then commit. Love it
The code in mini-blogs is becoming an issue and I don’t know if the rendering is going to fix anything once I figure that out. I think a better idea is just to use longer code blocks and just have comments in the code. That should make it a bit easier. Once I get to an obvious stopping place I will go back to text.
When I want to go back and change the code, Ill just comment it out. That at least keeps some sort of record, and i dont have to re run things.
Of the 4 things I set out to do I did 3 of them. Not terrible. I need to do some FKRB stuff tomorrow. But overall a good day.
Ok. So the plan for today is three fold. First, I want to do a paper summary on McMillan 2001. This is the subcenter identification paper that I will be extending with dynamics. After that, I want to work on some FKRB stuff. Probably get a data creation and estimation function going. I think that the data creation function is going to be the first Faster Function Friday. Third, the zbp scrape failed after 2004. I am having it run 2005 now, I think it might have just been a HTML timeout thing. If that finishes, Ill get it set up to run 2006:2018 overnight again. So thats the plan. Lets go.
I did not finish any of my three goals. But the paper summary is looking very good. Thank god for my blog or I definiately would not have remember to start the scrape again. Go check the McMillen summary. I still need to finishe the LWR portion, but its pretty good so far. I definitely have learned alot.
The eval stuff is really getting me. Some stuff runs, so doesnt
The goal for today is to get the site to a point where its fully functional, but still pretty ugly. That includes
After that I will mostly be working on the mini blog within the subcenter identification page. The next update should be saying that its all done, and the one after that will be on the mini-blog. Quick idea, mini blog H1 headers should appear as links in the main blog page.
I also want to append the myRender.R script to run the standard git process. So it would do all the rendering that it does now, stage all the changes, take the commit message as an argument, commit and push just from one click. Is this where I make my functions folder?
I should also mention that. I want to start doing weekly things. As I am currently talking to no one, its more of just a commitment device. I’m think the two things will be “GPU Function Friday” where I try to speed up a commonly used R function using the GPU, and Monday Paper Summaries. That gets me doing something over the weekend.
Back to site refinements.
I added a template for the Projects page, along with all the machinery to get the website to work. One thing I noticed is that the render now takes significantly longer. At some point, I’m going to need to have it check for changes in the Rmd files as compared to whats on Github before rendering. But, a more pressing issue is that it is doing double the work right now. The rmarkdown::render_site() creates a _site dir that has the entire site. Lets see what happens if I remove that now that I have a better idea of whats going on. The other thing is that it is rendering the templates, which obviously doesn’t need to happen, so I wrote a quick grepl to stop that. Lets see what happens
Everything is working. I have a couple commits that aren’t assigned to a blog post (I’m gonna start calling these unlabled commits) just to get it running. I had /Project/ instead of /Projects/ in the link for the projects.html page. Another thing I learned last night after reading about how the site updates is that my browser is caching the page and loading that instead of the newest version. If I want to see the newest version right away, I have to clear my browsers cache. Or I can just wait until it clears itself. This should appear along with the Mini-Blog and replication of the subcenter identification so lets go!
I got to working on the subcenter project and realized that I dont have a way to get around within the projects page. I think that the projects pages will get a secondary navigation bar that lets you cycle through the working paper, working slides, replication, and mini-blog.
I also have no idea how the working slides are going to work, but thats a problem for a different day. Lets get to work.
lol. I am trying some fixes and I did them on the miniblog.rmd side. But I dont have a way of checking that. So lets do it to the replicaiton.rmd, which actually displays. Lets go
It worked! I just want to add some pipes “|” to seperate everything like the traditional nav bar.
I also want to mention two things. When I am doing commits to both this blog and a mini blog, I should be saying which one it is. But when I do both that can be kinda tricky. For the most part, I should just be commiting more before changing to other things. But when Im still doing a lot of site refinement, its gonna get messy. So bear with me and I’ll try to make it somewhat intelligible
Second, I had a good idea in the mini-blog for the muni project. My replication was getting pretty stream of conscious-y, and I was trying to define the boundaries between a mini blog and a replication. What I landed on is that a replication can (and should) be edited while the blog, whether is mini or big, should not. So thats gonna be the boundary between those two areas for now. I would imagine that any traditional segment that Id have on the blog, like GPU Function Friday would be editted. So maybe that gets its own section like the Projects and Paper Summaries? I don’t know yet, lets find out. Back to work
It didn’t work. I have the working directory, but apparently the internet wants links to be actual urls not the location of the files on my local computer. Weird. Well lets fix that and get it to work. Maybe a little bit more debugging instead of just looking at it and saying its good would be warrented next time.
Mini update. I had WorkingSlide.html instead of WorkingSlides.html. This should be the last bug fix, for this. Lets go.
Well this little detour has taught me something. I need to be doing these site changes via a program instead of by hand and with templates. I basically need to be able to just call a function at the top of every page that will spit out the links to everything instead of changing it by hand. But thats defininately not what I need to be doing now. Thats a at home on my own time project.
Am I basically just making CSS? Yeah probably. But at least its my CSS and I know what its doing. I’d imagine that actuall CSS is better for this, but that seems like a something for a MUCH later date.
Ok, finally, back to work. Lets go
What happens when I have something that takes a while to run? I don’t want it to run everytime i recompile. Making sure only the current project recompiles is something on the todo list, but this is within that problem.
Obvious solution would be the run it, save it in a RDS, comment out the code block, and then load it. But thats very ugly and requires commenting and uncommenting. I think a better solution is to make everything a function. I can then either just comment out the function call, or I can write some if statements around it at the top. I like this solution and Im gonna implement it in the muniProject. I also need to figure out how the filing system is actually gonna work, but I think getting into the thick of it will tell me the best way. With that said, Ill report back from the thick of it.
GPU Function Friday is pretty restrictive. My thought at lunch was that it should be “Faster Function Friday”. This incorporates a lot of different ways that I can make a function faster, instead of just using the GPU. Im gonna update the about page quick, and then carry on.
I just commited and looked at the mini-blog. Thats a lot of work going on. Also, the no eval worked but it propagated through the entire system. Id imagine I need to turn it on for each thing? Idk. Burn that bridge when we get to it.
Calling it for the day at work. The replication code is take forever to run, so I am going to commit and then let it run over night. Were back at it tomorrow.
I am trying to get this site running. I’m hosting it on github pages through a Rproject. I like the simplicity of it so far, but the things I have read online have been suggesting a blogdown for larger projects. But I think I want to be the one whose doing all the background work, instead of relying on some blackbox to do it. Well, I am already relying on the blackbox of Rmarkdown, but I think I get that better.
Alright, the plan for the rest of the day is to get my municipal project onto here. What does that look like.
Lets go!
I think I want to document heavily, and to get that done, lets get a little formalism on how to do that. I imagine this isnt really where I should be doing stuff like this when working on a project, that should be on the diary page of that project, but this is like the diary of the website, so the rambling should go here. Lets say that all dated diary posts start with two hashtags and then any subsection start with 3. Lets see how this works in read time.
To render the site, save everything, rmarkdown::render_site(), commit and push.
2+2## [1] 4print(1+1)## [1] 2Lets add some more stucutre before working on the projects. I want the files system that is running this website to mimic the website. So the only thing that is going to go in the main part of the file is the index.rmd that contains the text of the front page, and the _site.yaml file that controls the formatting. Everything else goes into a folder called NavBar
In NavBar about.rmd and blog.rmd can hang out by themseleves because tthey are just static pages. But summaries and projects need additional filing. So in NavBar, there blog.rmd, about.rmd, a Projects Folder, and a Summaries Folder. We are gonna focus on the Projects folder, but it should be similar to summaries.
Within the Projects folder, we have the projects.rmd. This need to list Links to all the actual projects. Right now I have three. each with its own folder. Again lets work with MuniProject, but the others should be similar.
We want the 4 Files in here, working paper, working_slides, replication, and diary. Lets think about what the url should look like, which will hopefully get us a little closer to a good structure.
Let ~ denote chasewiedemann.github.io. So what do we want. We want to click on ~/Projects and see a list of Projects. We Then Want to click on ~/Projects/MuniProject/ and see a the 4 ingredients. Ok. Lets get ~/Projects Working. I lied. I want to do this for the MuniProject and the GridEstimators Project as an example to make sure it works for multiple. I should probably commit.
I made a bunch of chagnes to the files structure, but didnt tell my _site.yml file about it. But fortunately, it seems like my output_dir docs file is really good at mimicing my file structure. I have everything fixed and will commit now.
When I say I am going to commit after immediately describing something. I will make the commit message the next highest header, underscore, and then the date. For example, I am going to commit now. the commit message will be strcuture note_081124
It didn’t work. I guess I have to figure out what this output_dir is actually doing. Or do I need to make them html? Lets see if I can find a link
https://bookdown.org/yihui/rmarkdown/rmarkdown-site.html
This guy is saying (Right about 10.5.2) that all the .rmd or .md files in the ROOT of the directory will be rendered into HTML. Mine are in folders.
Looking more into the site, it seems like the entire reason that people recommend blogdown is for this reason. It will only render the .rmd into html that are on the front page.
I think what I need to do is output everything to a html_file. Im gonna try that with this the projects.rmd, about.rmd, projects.rmd page
What I need to do is write a script that goes through all of my rmds. renders them, and creates the html doc.
TEsting on the terminal now. Lets see if i can get blog.rmd to create NavBar/blog.html
That Worked. So What I need to Do is create my own script that will search through the NavBar director, and render all the .rmd files in it. lets go!!
Ok heres the script. its title myRender.R and is found in the main files
library(tools)
all_files = list.files(paste0(getwd()),recursive = T)
toRender = all_files[file_ext(all_files) == "Rmd"]
for (files in toRender) {
rmarkdown::render(files)
}
rmarkdown::render_site()All it does is go into the NavBar directory, find all the files that end in .rmd, and then render them. It then runs the render_site() command that seems to be doing a lot of the backend work that I dont fully understand yet. One potential downside I see is that this will render literally everything. That could be an issue if I have a large project somewhere. I think I can probably just go and check to see if its different from the github page, and then not render it if it hasnt changed, but thats an issue for another day. Lets commit by running myRender in the terminal
I got it working. Sometimes i get a weird index.lock file that I can only delete once I’m outside of R. Theres obviously more going on here than I comprehend, but I think I’m getting the hang of it. I just updated the home page, lets see if that works
I got everything running exactly as I have written it. The only issue is that the exact way I have written it removes all the links to other files. That means that once you are on the about page, theres no way to get back to index or projects. Lets fix that now. What i am trying first is just to add a navbar like in the _site.yml file into the about.rmd. Hopefully when this renders, it will add a navbar. Lets commit and check. I forgot to run myRender first, so lets find out if I messed everything up
Two things. First, I should be need to rethink my commit messages. Just Going up 1 doesnt make sense. So What Im going to do is to go up 2 and have both. So the day post will be a single hashtag, and then a topic will be a double. Then withing that topic i will be doing updates with a third hastag. So this one will be This Script_Update 3 081124
The other thing is that I don’t know if my changes do anything. Im gonna need to remember to add something like “test TExt” to the bottom of a page Im testing
Its not updating anything. I can see on the github page that it went from pending to deployed. Im gonna check again before commiting the next changes. What I did is change the _site.yml to “doc/NavBar/~” instead of “NavBar/~”. Again, I think this is more because I don’t know what render_site() is actually doing, but this seems like an easy fix for now
Good thing I checked, the text loaded, but the about.html still doesn’t have a navbar. Lets see if what I was previously planning on doing has a fix. I just have to be patient, and add a new Test Text
I think I know what to do here. My last commit showed changes to the NavBar folder, but not to the docs folder. I need to port over my rendered stuff into the docs file. Heres the script
library(tools)
all_files = list.files(paste0(getwd()),recursive = T)
toRender = all_files[file_ext(all_files) == "Rmd"]
toRender = toRender[toRender != "index.Rmd"] ### Because Render Site will do this
for (files in toRender) {
rmarkdown::render(files)
file_name = gsub(file_ext(file),"",file)
file.copy(from = paste0(file_name,"html"),paste0("docs/",file_name,"html"))
}
rmarkdown::render_site()Not very elegant, but it should work.
I scraped all that. The navBar idea is a non-starter at this point. I looked at the html code that was built, and hopefully at some point it isnt giberish to me, but for now its incomprehensible. Lets just build it statically. You should be able to get back home with a button click from every page. Thats gonna just need to be a template. I think thats what I will need to eventually do. Create a template directory that I can just add from. But lets start a new top and work on getting a link at the top of the page from the about page to the home page. This will create a pattern that we can replicate
Lets look at the internet for help. It seems pretty easy. Heres there example RStudio. Lets add something similar to the about.rmd, but its gonna take us Home. I have no clue what its gonna look like, and i just added the url of the home page to it
It worked, but now I want to do something a little more basic. Lets get rid of the pre-built navbar in favor of more control. I removed all the stuff from the _site.yml file. Lets get links to all the other stuff into the index.rmd file
We ran into some issues. Without the additional help from the black box, we never create a index.html page. I had previously skipped over it, but now we need it. I commited before writing this under How to get Home unlabled 1. Lets see if it works
It still is failing its build. Im gonna look at the index.html that it loads in a local web browser to make sure its not something dumn about me doing links wrong. A possible problem is that when I run rmarkdown::render_site(), it is deleting some of my stuff. Lets try going full manual
I think I found a (the?) issue. In the Setting page of Github, I had it building from a file called docs because thats how the tutorial that I read told me to do it. Now that I am straying from the source, I need to make sure I dont have any of those old relics in there. Lets see if that fixes it.
It works. This is as basic as I can get this site, and I think that probably for the best (for now). I think Im going to get all the links into the pages, figure out the projects page, and then call it a day. Next commit will be all the Site links, then onto the projects page
I’m not sure if I’m not being patient enough to let the changes go through, or if its fixed, or if I just broke it. Im going to go update every page and run it how I have it now. Then Ill let it sit for awhile and check if it ever updates
Unless it fails. It works, just takes a while for the site to catch up. I foresee this being an even larger issue when I start making bigger changes. But another problem for another day. Im gonna have the myRender script take out the _site dir to clean everything up. Ive also updated the checks on each page to include a 2