This work aims at evaluating the usability of remote sensing RGB imagery by an Unmanned Aerial Vehicle (UAV) in assessing wheat drought status. A UAV survey is conducted to collect high-resolution RGB imageries by using DJI S1000 for the experimental wheat fields of Gucheng town, Heibei Province, China. The soil moisture for different plots of the experimental filed is kept at an approximately constant level for the whole growing season in a well controlled environment, where field measurements are performed just after the UAV survey to obtain the soil water content for each plot. A machine learning based wheat drought assessment framework is proposed in this work. In the proposed framework, wheat pixels are first segmented from the soil background using the classical normalized excess green index (NExG). Rather than using pixel-wise classification, a pixel square of appropriate dimension is defined as the samples, based on which various features are extracted for the wheat pixels including statistical features and spectral index features. Different classification algorithms are experimented to identify a suitable one in terms of classification accuracy and computation time. It is discovered that Support Vector Machine with Gaussian kernel can obtain an accuracy over 90%, which demonstrates the usefulness of RGB imagery in wheat drought assessment.