Math subject GRE 3768 problem 3

If you see any mistakes, or have suggestions / alternative methods of solutions that may help people study, please add them to the comments. Good luck with the test, everyone!