All I said was set up appointment in calendar and it took me here. ... NOT COMPLICATED AT ALL

 

Actually I think the OP is correct. I believe he is looking for a simple easy voice commands experience like "siri" that you can speak to the phone and it will do exactly what you say. Telling it to make an appointment only opens up the calendar but you have to input everything else manually. In siri you can command it to open up the calendar and create, add & delete entries on the apt and close itself out. OK Google does not provide this type of experience.

Is there a way to use voice commands to direct the phone to perform task instead of manually doing them?

Ok google seems to be fine for a limited things but not to really do much if you are using voice commands

 

I use Samsung's S Voice on my S4, and it works quite well. Personally I prefer it to Google Voice.
I presume the N4 also has S Voice on it, you might give that a try.

 

So. ....I tried S voice on my beautiful note 4 gave it the same command....it works great.

 

Then you haven't learned how to use it yet. It does the same stuff Siri does.

 

I've been using both Google and S-Voice for different voice searches. They both seem to work well, but neither of them is perfect (not that I expected either of them to be).

 

OP, you have to say it in a specific order. "Set reminder [reminder action] for tomorrow morning" or substitute the day of the week or date. You can also say a specific time if you prefer that.

Google search OK Google commands, and/or check out YouTube.