I will discuss the behavior of odi startscen.sh/bat command. Often, we use this command to execute a scenario from our custom shell/batch scripts or might even add it to our workflow engine.
The syntax being:
startscen.sh scn_load_data 001 GLOBAL -NAME=odiSchedAgent001
However, there is an important point to know the underlying functioning of the startscen command. This command forks out a new agent process that is local to the machine on which the command was started.
The startscen.sh has a parameter -NAME=<agentName> which naturally makes us assume that this agent will be used to execute the scenario. However, this parameter is just used to store the agent name in the logs of the operator. It has nothing to do with using that agent to execute the scenario.
Another obvious side effect is that the OS execution privileges of the user who initiated the startscen command are used to generate log files or any other OS files, which may be different from the execution privileges of the remote agent that you intended to use.
Consider a small test case:
(1) Use Unix account "A" to start a background agent process by agentscheduler.sh.
agentscheduler.sh -PORT=20910 -NAME=odiSchedAgent001
(2) On the same Unix machine, use Unix account "B" to execute the scenario "scn_load_data" startscen.sh
startscen.sh scn_load_data 001 GLOBAL -NAME=odiSchedAgent001
However, in step (2), it would fork a new agent process to execute the scenario instead of using the existing agent process started in (1). You can verify this by scanning the processes.
On Unix, you can check that using "ps -ef" and see that a new process would be initiated.
Even if you use same Unix account "A" to execute startscen.sh, it would still fork a new agent process.
This may be a problem. As I said earlier - that the permissions of the user who initiated the agent vs the user who initiated the startscen command would differ.
However, there are more cases in which this will be a Bigger problem.
Consider a scenario in which you have 2 Unix systems - System-A and System-B. System-A is running the agent process and the System-B is being used to kick-off the scenario using startscen command. If there are any OS based activities done by the scenario, then they would be done on the file-system of System-B, whereas you wanted those to done on System-A (after all thats where the agent is running. Or so you thought)
Now the Good part... The Solution :
Use the startcmd.sh instead of startscen.sh
The syntax is
The syntax being:
startscen.sh scn_load_data 001 GLOBAL -NAME=odiSchedAgent001
However, there is an important point to know the underlying functioning of the startscen command. This command forks out a new agent process that is local to the machine on which the command was started.
The startscen.sh has a parameter -NAME=<agentName> which naturally makes us assume that this agent will be used to execute the scenario. However, this parameter is just used to store the agent name in the logs of the operator. It has nothing to do with using that agent to execute the scenario.
Another obvious side effect is that the OS execution privileges of the user who initiated the startscen command are used to generate log files or any other OS files, which may be different from the execution privileges of the remote agent that you intended to use.
Consider a small test case:
(1) Use Unix account "A" to start a background agent process by agentscheduler.sh.
agentscheduler.sh -PORT=20910 -NAME=odiSchedAgent001
(2) On the same Unix machine, use Unix account "B" to execute the scenario "scn_load_data" startscen.sh
startscen.sh scn_load_data 001 GLOBAL -NAME=odiSchedAgent001
However, in step (2), it would fork a new agent process to execute the scenario instead of using the existing agent process started in (1). You can verify this by scanning the processes.
On Unix, you can check that using "ps -ef" and see that a new process would be initiated.
Even if you use same Unix account "A" to execute startscen.sh, it would still fork a new agent process.
This may be a problem. As I said earlier - that the permissions of the user who initiated the agent vs the user who initiated the startscen command would differ.
However, there are more cases in which this will be a Bigger problem.
Consider a scenario in which you have 2 Unix systems - System-A and System-B. System-A is running the agent process and the System-B is being used to kick-off the scenario using startscen command. If there are any OS based activities done by the scenario, then they would be done on the file-system of System-B, whereas you wanted those to done on System-A (after all thats where the agent is running. Or so you thought)
Now the Good part... The Solution :
Use the startcmd.sh instead of startscen.sh
The syntax is
startcmd.sh OdiStartScen -SCEN_NAME=scn_load_data -SCEN_VERSION=001 -CONTEXT=GLOBAL -AGENT_CODE=odiSchedAgent001This command actually uses the remote agent specified in the parameter.
7 comments:
Very helpful post. Most appreciated.
Thank you. Kudos!!
Hi blogger,
Thats the best i could get on agent. However, I would like to know what if I execute the command
startcmd.sh OdiStartScen -SCEN_NAME=scn_load_data -SCEN_VERSION=001 -CONTEXT=GLOBAL
As you can note, I havent specified an agent. So will this fork a new local agent too?
Please reply to me on my email
k_anmol1592@hotmail.com
Anmol,
It would do no harm in trying to run the command.
I dont think this is going to execute anything. Whats the logic behind not specifying an agent_code ?
If you want to use the local agent, then go with odistartscen
Hi Ankit,
Thanks a lot for your response.
Actually the environment I am working in is quite restricted. I have been asked to write unix scripts that check whether ODI is up (by checking if the local agent is running or not :|) and then executing the scenario we are trying to execute. I am not sure how to ping a local agent from a Unix Command line.
My understanding is that everytime a job is dispatched with a local agent, a new agent instance starts for its execution. A local agent, unlike a standalone/scheduler agent is not always up and running.
PS : The command "startcmd.sh OdiStartScen -SCEN_NAME=scn_load_data -SCEN_VERSION=001 -CONTEXT=GLOBAL"
will execute with an internal agent as we are already doing this here.
Your understanding about a new instance being created while using local agent is correct.
However, there is nothing like a "local agent" running all the time.
So, this is not something that can be checked by pinging in Unix scripts.
It is only created when you invoke startCmd or StartScen. After the scenarios completes, the local agent is destroyed as well. The local agent is alive and available only for that specific session.
ODI is never "up" by using a local agent. The only way ODI can be "up" is by using a standalone/scheduler agent. This is a process that keeps running on the server. This process can be tracked by ps -ef and if its available, then startcmd can be executed.
Also, there is another util inside ODI (OdiPingAgent) to check if a particular agent is available or not.
Hi Ankit,
Thanks a lot. I conducted a POC on the so called 'local agent' and came to the same conclusion. A local is not a agent. It is simply an execution unit started by ODI to execute the scenario, which terminates when the scenario has been executed(successfully or not doesnt matter)
So now I have proposed to call a dummy scenario, using the "startcmd.sh OdiStartScen" command. This is the best way to check the 'local agent' and to check if all ODI configurations are intact to handle an incoming execution request.
Cheers.
Thanks Kudos, very helful!
Post a Comment