Shari, the only book you need to worry about right now is the ICD-10-CM code book itself. That is the only one of your current books that you will be taking to certification exams. Later, it will apply to the CPT, HCPCS, and ICD-10-PCS code books.
You CAN write in the books. You just shouldn't write very much. You definitely should not write chunks of guidelines or paragraphs from other books. Try to keep what you do write small, neat, and discreet. You may highlight, as well, but keep that highlighting professional, neat, minimal, and discreet. No big swaths of scribble highlighting in glaring colors all over, for example.
You will see people on the Internet saying that you can write anything you want in your code books. They are thinking of the CPC exam. Unlike them, you have to plan for the CCS exam, as well. Not all coders take that, nor do they know anything about it. The CCS exam does not allow as much writing. The proctors at some test centers will turn you away if you have too much writing and we even had one student who was told that it was a good thing she only had highlighting in one color, because two colors was not allowed. (AHIMA says nothing about colors. I think that that proctor was thinking of the rules for another exam that that Pearson Vue administers.)
Just by way of advice, I would recommend that you do not make very many notes or highlights at all in either your ICD or CPT manuals until you get into Module 3. Until then, write what you want on a sticky note and paste it on the page. You will find that by the third module you will no longer need those notes and you might even wonder why you made them to begin with. Prior to your CPC exam, you can go page by page through the books neatly writing the notes that you truly need into the books and removing all the stickies. (Stickies and loose, glued, or stapled papers are not allowed on either exam.)
You are going to see comments online about tabs. Many coders will tell you that you absolutely MUST apply tabs to your books or you will FAIL the exams. This is nonsense. Tabs don't really help you one way or the other and might actually slow you down. For the CPC exam, you may use tabs but they must be permanently affixed. For the CCS exam, the Pearson Vue proctors might make you tear the tabs right off the pages or forfeit the exam. If you got used to using the tabs, suddenly not having them will cost you points on the exam. That's why we do not recommend that students tab. There's really no need for it.
You will also see comments online about special methods that you MUST use to circle related code groups, highlight critical information, and underline other information OR YOU WILL FAIL. Various schools impart the secrets in videos on YouTube intended to sell you on the necessity of taking their course rather than the one you are currently taking. You do not need those methods. Andrews students have a 97%+ pass rate on those exams and many of them have not one single thing written in any of their code books. Students who take a course that lasts only four months and offers nothing more than tips and tricks for passing exams do need those methods. Your course lasts far longer and contains more than four times the material. You will have so much exposure to coding and your books that you will not require those kinds of tricks to pass. Your instructor is a better source of information than YouTube videos, so you might want to think twice before you use them.