“Memory fades, memory adjusts, memory conforms to what we think we remember.”

― Joan Didion, Blue Nights

learned some new ideas about LLM research. the idea of instruction pretraining, where you use LLMs to generate synthetic instruction data, and use it during pretraining. you can combine instruction tuning with to have LLMs reason better i.e. in solving math problems. small, high-quality datasets can outperform large ones. still unsure how this is different from fine-tuning, it's more about having LLMs follow instructions? and there's also midtraining, which is about improving the capability of LLMs, rather than usability?

not having a practicum now is coloring my every moment in sf. it's a pervasive mood, to feel like you're at the very top, and fall right to the bottom, the next second.

it's another reminder that the world doesn't revolve around you and you can't be too attached to things. when they're taken away from you, and you have to fight to create a different path, your own path.

i could almost see a path before this, now it's all blurry again. and maybe that's a good thing.

dryer was broken (taken away?) so we went to north beach, near washington square, to do our laundry. read a bi of joan didion's let me tell you what i mean. i found the foreword to be flowery and hard to understand, maybe i need to expand my vocabulary and read more.

two quotes about writing:

The peculiarity of being a writer is that the entire enterprise involves the mortal humiliation of seeing one's own words in print.

Had I been blessed with even limited access to my own mind there would have been no reason to write. I write entirely to find out what I'm thinking, what I'm looking at, what I see and what it means. What I want and what I fear.