a3nm's blog

Why research feels really hard at first

— updated

I have tried to do research for about two and a half years now. I haven't quite figured it out yet, but I have understood a few things that I want to write down, before they seem so obvious that I don't even remember I had to notice them.

The point of this post is to explain for which reasons doing research is difficult, especially when you are getting started in the field: say, doing a research internship, or just starting a PhD. So, if your experience is not entirely positive, this may help you identify possible causes, and understand how much they may improve with time. Of course, everyone's perception is different, and maybe you legitimately dislike research, so it is difficult to tell how much my advice applies to your specific case; still, I hope it gives you food for thought. Of course, parts of what I write apply to any activity where you are getting started, not just research, but even the general points may not be immediately obvious if you aren't warned about them.

Research is intrinsically hard

You need to keep this is mind: research is actually difficult, so it is normal that you feel that way. It is normal that things progress slowly, in fits and starts, that you sometimes get discouraged, and often procrastinate. There are several reasons why real research is hard:

  • It must be new, so you need to find things ideas that haven't been tried out, and solutions that no one else saw yet.
  • No one knows how to solve your problems, so no one can really help you.
  • No one knows whether things will work, so there is a fair chance your hard work will not pay off no matter what, and it usually takes many iterations to solve anything.
  • No one directly needs you to solve the problem. No one's directly looking at you and waiting for you to do the job. If you are not making progress, probably no one cares, except maybe your advisor, and then only because they are your advisor, probably not because they care intrinsically about the problem.

You may feel really bad if you think of research as just another job, and compare yourself to someone with a "normal job" that does not face the same challenges. When your job is to do something that you're proficient at, that you've done many times and that everyone understands, and where people directly need you and you just can't not do it, it's a lot easier, and you can't procrastinate as much.

It may sound trite, but I remember it wasn't obvious to me at first: researchers are essentially like artists. The only difference is that research looks more like an office job, both in terms of having an actual office, and of being paid a fixed salary. However, if you think of it as art, it seems more normal that you are not continuously productive for 8 hours every single day. You need inspiration, it takes a lot of energy because you are creating something new, and there are false starts and ideas that turn out not to work out well in practice.

You are doing only research

As a young researcher, you probably do not have a lot of diverse things to do. You don't have a backlog of email to reply to, things to read, things to write about, papers to review. More significantly, you probably do not have many administrative obligations or teaching duties. In fact, the very moment where you have the most time to devote to actual research is paradoxically when you're the least qualified to do it, while the experienced researchers are swamped by other duties. It makes little sense, but that's life.

It may seem strange to claim that filling administrative forms and teaching classes can make research easier, but I think it is true, up to a certain proportion. It is good to have some part of your job where tasks are gratifying because they can be completed without turning your brain on (trivial tasks) or they make you feel useful to others (teaching). If your entire professional life is research, you feel bad if you lost one week because you messed up on something or didn't get any inspiration; but if you have also managed to do other things that aren't research, it is more palatable. Likewise, if you did research for 4 hours and goofed off for 4 hours on a given day, you feel bad. If you did 4 hours research and 4 hours mindlessly replying to tons of trivial email, you feel extra productive! So you may feel better once you get to spend more time being efficient on professional things that are not hard research.

You have only one project

Not only are you doing mostly research, you probably have only one project, for example, your internship or PhD topic. Of course having too many things to do can put more pressure on you and make you lose time and energy in mental context switches. But on the other hand, having multiple projects allows you to switch from one to the other when you get bored, and more importantly allows you to move on temporarily to something else while your current project is stuck (while you wait for your collaborators to make progress, wait for your unconscious mind to come up with a new idea, etc.).

Devoting all your energy to a single thing makes you extremely vulnerable if it does not go well. In research as in life in general, it is a bad idea to put all your eggs in one basket, and you will feel more stable if your self-esteem depends on multiple independent things.

You haven't chosen your project wisely

Not only are you probably working on one project, you are also probably working on your first project. Another huge perk of having been around for longer is that you had more time to discover what you like, and do more of it, and less of stuff that you don't like. As time passes you will figure out which tasks you prefer, and which themes. You will switch to different collaborators (or, maybe, supervisors) if things aren't working out well with the current ones. Your first project is only an entry point, so on average what you will do afterwards will tend to suit you more, so it will probably be more pleasant, which will make it easier to be productive on it.

You are mostly working alone

Except from occasional supervision by your advisor, you may be the only person involved on your project. If so, be aware that this is not the usual way to do research. In my field at least, the vast majority of research happens as collaborations within small groups of people.

When you are working on a project with other people (as peers, not as supervisors), it's much harder to lose steam, because they are multiple people pushing the project forward, a lower chance that they are all discouraged simultaneously, and you do not want to let them down. (The energy that you get from someone supervising you is not the same as that of someone who works with you on a equal level; and feeling responsible for your commitments to peers is not the same thing as vertical accountability to your supervisor.)

The advantages of a successful collaboration are that it mathematically splits the workload, and gives you access to the skills of your collaborators for areas you may be unfamiliar with; but more importantly, it sets up a situation where the collaborators are all relying on each other, so the project moves forward.

Another aspect is that some tasks are easier to perform with other people. I find it much easier to make progress on complex issues by discussing them with someone else: of course I also need to think on my own, but I find that discussions helps to flesh ideas out, and sometimes leads to new insights (also, the rubber duck effect applies). For writing, it's easier to proofread someone else's prose than your own because it is hard to spot your own mistakes, and it's easier to write if you know that someone else will tidy things up behind you. A scheme that works well is to have multiple people perform "passes" to successively write and edit the text: start with a very rough draft (already hard to spit out, but easier because it can really be crap), and then correct, correct, until you converge. You can emulate this on your own but it's harder because you need to go back to your own prose and improve something that you wrote just before.

People have invested less in you

If you are "just" an intern who recently got started, your advisor has not "invested" a lot in you yet, in comparison with more important time and money investments like ongoing PhDs or collaborations which are already successful. It is also for this reason that you are probably working on your own: no one has committed to working with you yet.

Your advisor may not be doing conscious accounting about the importance of their commitments, but, as they accumulate, those that will get sacrificed are usually the ones with the least sunken costs, the most distant prospects of payoff, and the smallest embarrassment in case of failure. It is sad and wasteful to inadequately supervise an intern; it is a professional failure and possible future embarrassment if you let down your PhD student, whom you've supervised for three years, and who may be staying in the field later.

There is little cure to this except time and work. As time passes and you stick around, you build up a reputation for reliability. There is a "rich get richer" effect: the more successful people are more reputable, so get more commitments, so get better chances to build up their skills, knowledge and contacts, which make them more successful still.

You don't know existing work

Another hard thing in initial research is that you have to get acquainted with all that's already been done in your field. In some areas, this can mean that you need to spend months or years reading up the relevant literature, before you can hope to contribute anything meaningful or new to the discussion. Remember that the discussion is between people who have been reading each other's work for many years. If you are a newcomer, you have a lot of catching up to do.

More locally, if your research project has been started before you, you have to understand what people have been doing before you within that specific project, which is also hard. For instance, in computer science, you may have to get acquainted with other people's code, which is much harder and duller than starting out on your own cool new project from scratch.

There are advantages to being a newcomer, though: as your perspective is shaped more by the current state of things and less by the whole history of the discussion, your mind is freer to contribute new insights. So don't worry, you will not always remain in the shade of the people who were here before you.

Your skill and confidence will improve

A very general thing: when you start doing research, you are less skilled at it, so it feels even harder; when you become more skilled, it feels less hard. Now, you can never be entirely proficient at research: as soon as you really master something, you have to move on to something else. Yet you eventually practice meta-skills such as understanding things, thinking about them, organizing your time, etc. You accumulate a culture about your field (see previous point). Last, you exercise a bunch of tangential skills: mastering your tools, your computer, LaTeX, etc. So you become more skilled as time passes, and the tasks becomes less hard, and less daunting.

Also, as you get started, you are not so confident that you will be able to achieve anything in research. As time passes, you achieve a few things, and so you start becoming more confident, because you have objective proofs that you are competent. This point is even more general and a little bit meta, but it can be a huge deal, because lack of confidence and impostor syndrome is a huge problem for a lot of people in research. It's normal, it usually gets better, and in any case you normally get used to it after a while.

comments welcome at a3nm<REMOVETHIS>@a3nm.net