is it possible to create new custom group action ?
Sorry, it is not possible.
The few group actions that are there are hardcoded.
Depending on what you are trying to achive I have added 'group actions' by creating custom workflows that will execute the required action against all objects in a particular group. I then just put a link to this workflow on the particular page that I want to execute it against.