We consider the general nonlinear regression problem . A
survey of some classical methods and stochastic approximation procedures for
estimating is first given. We solve the nonlinear regression problem by
considering the optimal stochastic approximation procedure by [3],[4]. This leads us
to introduce a new procedure , called "Stochastic Approximation Iterative Least
Square Procedure" SA -ILS procedure. The new procedure is applied to a number of
nonlinear regression models. We report on the results of a simulation investigation
which indicate that the new procedure is highly efficient with respect to the number
of observations required to obtain the parameter estimates for given regression
problem.