Some agricultural tasks performed in a crop field consist of applying chemical treatment to the plants. In order to automate these tasks, a vision system and a treating device can be used as a sensing device, mounted at the rear of an agricultural vehicle. The vision system tracks each plant in a sequence of images. With this arrangement the position of each plant with respect to the treating device can be derived which specifies when and where to apply the chemical spray. A map of the field has to be recovered in the local area between the vision sensor and the treatment device. This implies to recover the motion parameters of the vehicle, to record the trajectory of the vehicle, and to place the plants in the map. A method to identify the plants, a shape description to match them and a tracking strategy are presented. The motion parameters are recovered from the plant correspondences, using a method to find the motion of a planar patch. A Kalman filter is used to integrate the different observations of each plant and to place them on the map. Results with real image sequences are presented, including zigzag and rotational movement.