1. Download our Official Android App: Forums for Android!

Automated Keyboard and Mouse actions

Discussion in 'Android Apps & Games' started by Pente, Feb 3, 2014.

  1. Pente

    Pente Lurker
    Thread Starter
    Rank:
    None
    Points:
    5
    Posts:
    4
    Joined:
    Feb 3, 2014

    Feb 3, 2014
    4
    0
    5
    Hi all.

    Not sure if this has been covered, and I have been searching for it with no luck.

    I'm wondering if there is a way to automate mouse and keyboard actions so that events such as logins etc. can be automated and initiated unattended or called by another app (even if it is just for known sites/tasks etc.). I'd be happy with having to 'record' a macro with mouse/keyboard actions and just have the macro be callable from something such as tasker.

    In particular, I have an app for my HVAC system that requires me to enter a password and hit enter in order to log in to the company's servers. From there I can make adjustments to my home heating/cooling system. I'd like to be able to automate this so that tasker or some such app could open the HVAC app, type in my password, and ideally move and click the mouse on the UI presented (it mimics my local thermostat input screen). The tasker trigger could be based on a text message from my home automation controller attached to temp. sensors, for example.

    I can't just use a username/password vault type app as I need to actually move the mouse and register clicks etc.

    Any help would be greatly appreciated.

    -Pente
     

    Advertisement

  2. Rukbat

    Rukbat Android Expert
    Rank:
    None
    Points:
    843
    Posts:
    10,392
    Joined:
    Jan 16, 2012

    Jan 16, 2012
    10,392
    2,894
    843
    Male
    Being retired and writing Tasker tasks.
    Fayetteville, NC, USA
    Are you writing an app to simulate the functions of the existing app (easier way) or trying to automate the existing app (more difficult way)?
     
  3. Pente

    Pente Lurker
    Thread Starter
    Rank:
    None
    Points:
    5
    Posts:
    4
    Joined:
    Feb 3, 2014

    Feb 3, 2014
    4
    0
    5
    Rukbat,

    Thanks for the reply.

    I was actually not planning to write an app (my coding skills are more than a tad rusty these days), and I'm not sure I could do it for this app anyways, the HVAC company has a pretty tight grip on their communication protocols etc. so I'm not sure one could even get a 'clone' app to even authenticate on their servers, at least, not without far more work than it is worth.

    On the windows side of things, there are apps that will record your mouse movement and keystrokes, creating self-contained macros (think batch file) that can then be run. The macros are basically agnostic of the actual application running in that, the macro will run (mouse moves, changes co-ords, any clicks and/or keystrokes are executed, regardless of whether or not the app which has focus is actually receptive to these mouse/keyboard actions. In this respect the macro is dumb, running in an over-lay if you will, regardless of what app is actually in focus.

    Think of it this way: the password text box is always at position x,y in my HVAC app, so, I'd like a macro, or event, that would open the HVAC app, move the mouse to co-ords x,y, left click (activate cursor in the textbox), enter my password into the text box, and then <enter/return> to complete the sign-in process. All of this would be autonomous, and unattended.

    If this can be done, it is then just a matter of making multiple macros for any/all such actions required ie. mouse-over and left-click on an onscreen button to increase temperature of thermostat.

    I hope this is more clear, sorry for initial confusion.

    Thanks in advance,

    Pente
     
  4. codesplice

    codesplice Elite Recognized Moderator
    Moderator
    Rank:
     #13
    Points:
    1,563
    Posts:
    8,716
    Joined:
    Oct 29, 2013

    Oct 29, 2013
    8,716
    10,100
    1,563
    Male
    SysAdmin
    Huntsville, AL
    You may be able to use Tasker in conjunction with the input tap shell command to create a script that would function like you describe. There's a pretty good write-up on that here: How to emulate touch input using Tasker and root - Pocketables

    If you do decide to go the Tasker route, let me know and I can move your thread into the Tasker subforum where lots of neckbeards smart people are ready and waiting to assist you.
     
  5. Pente

    Pente Lurker
    Thread Starter
    Rank:
    None
    Points:
    5
    Posts:
    4
    Joined:
    Feb 3, 2014

    Feb 3, 2014
    4
    0
    5
    Codesplice...

    Tyvm! That is an excellent link and from a quick read-through, exactly what I need for the mouse movement etc. so all I'd need to do is figure out the keystrokes...hopefully there is something already baked-in for that too (I'm yet to really jump into Tasker, but will be starting soon enough)

    I will take you up on your offer, and thank you in advance, for moving this thread to the tasker subform...I'm certain you are correct about that being the best place to continue with this.

    -Pente
     
  6. codesplice

    codesplice Elite Recognized Moderator
    Moderator
    Rank:
     #13
    Points:
    1,563
    Posts:
    8,716
    Joined:
    Oct 29, 2013

    Oct 29, 2013
    8,716
    10,100
    1,563
    Male
    SysAdmin
    Huntsville, AL
    There's some discussion at the Tasker Google Group on that topic, but I haven't perused it enough to know for sure if it will work for you. It may at least get you pointed in a helpful direction. :)

    Done :D
     
    D-U-R-X and Pente like this.
  7. KitWasHere

    KitWasHere Well-Known Member
    Rank:
    None
    Points:
    53
    Posts:
    109
    Joined:
    Mar 31, 2013

    Mar 31, 2013
    109
    39
    53
    There is a "Type" action under "Input" you could try. Don't use it myself but other users report it as being slow. You need root for that, as well as the "input tap x y" command posted above.
     
    Pente and codesplice like this.
  8. Pente

    Pente Lurker
    Thread Starter
    Rank:
    None
    Points:
    5
    Posts:
    4
    Joined:
    Feb 3, 2014

    Feb 3, 2014
    4
    0
    5
    Kit and Code...

    Thanks for the continuing help and suggestions.

    I will certainly look into that forum Code, and the 'type' action. Speed won't really be an issue as long as it isn't glacier-like, as most of my intended plans won't be of a pressing nature.

    Much appreciation!

    -Pente
     

Share This Page

Loading...