No matter what services your M-file provides, good citizenship requires that it be portable to all Matlab-5 platforms. When not, prospective users need to be told the known limitations of the software, plus steps that can be taken to gain maximum benefit from it.
For example, P-code files in Matlab 5.0/5.1 do not work in Matlab 5.2+. In the case of object-oriented programming, on the other hand, some of the rules changed, but Matlab-5.3 OOP is backwardly compatible with earlier versions, though not the other way around.
Concepts that need to be addressed include the various styles of pathnames and newline characters. The "computer", "filesep", and "sprintf('\n')" functions are helpful here.
Some workers are still using Matlab-4, for which the Matlab-5 linguistic extensions are not an option. These advances include "{...}" notation for cell-arrays, "." notation for structs, and the entire arena of class-oriented (aka object-oriented) programming. Many M-file functions that were new with Matlab-5 are actually compatible with Matlab-4, but their usage in the latter raises thorny licensing issues. Updating to Matlab-5 is well worth the nominal expense, so it would make sense to encourage same.
(EDITORIAL ON -- Although Macintosh Matlab-5.2 is more advanced that its PC and Unix counterparts, the MathWorks has decided to "stabilize" the Macintosh version, which is another way of saying "drop dead". Being officially "dead" is not really so bad yet, but it's not something to look forward to in the long run. Meanwhile, PC and Unix Matlab-5.3 includes some volumetric graphics -- easy enough to do on a Macintosh, except that the new software requires a Mex-file that is deliberately not available for the latter. Programmers expecting compatibility with Macintoshes will need to pay attention to this situation. -- EDITORIAL OFF)
When an M-file is called without any arguments, it should show its "help", at minimum. Then, if appropriate, it should perform a demonstration with default arguments, or it should ask for input, assuming any is needed to perform a useful task. Commonly, one or more filenames are needed, such as might be requested through "uigetfile()" or "uiputfile()".
With a set of sensible default arguments, an M-file can easily perform a self-demonstration whenever it is called with the word "demo". The "help" should be shown as well, so that the user can gain appreciation for the typical usage and behavior of the routine.
The M-file should be prepared to deal with a variable number of output arguments, possibly through the "varargout" mechanism. In such a situation, it is important to remember that the "varargout" cell-array must be pre-allocated within the M-file, since Matlab does not provide it automatically.
As with output arguments, the M-file must be prepared to deal with a variable number of input arguments as well, perhaps through the "varargin" mechanism.
Matlab passes command-line arguments as strings into the targetted function. This handy mechanism is often exactly what is required, such as when a filename is needed. However, one need not stop there, since even the names of existing variables might be passed into the routine as strings, whereupon they need to be evaluated in the proper context of the caller. Within the targetted routine, the "ischar" function can be used to verify that an argument is a string. If so, the string can be evaluated in the context of either the "caller" or the "base" workspace, by using the "evalin()" function.
Assignments can be made in the workspace of either the "caller" or the "base", using "assignin()" function. This mechanism is especially useful when a silent assignment is desired, perhaps to "ans", as in "assignin('caller', 'ans', x)". The assignment need not be to a pre-existing variable.
Expressions can be evaluated in the context of the "caller" or the "base", using the "evalin()" function. This mechanism is especially useful when the user needs to have the evaluation returned to the current routine, as is usually the case.
M-files need some documentation, since they might need to be repaired or explained. It is easy to ignore this phase of programming, but equally easy to overdo it. There are no general rules to follow, except that some thought needs to be given to the likelihood that others will be using your M-files in the future.
Common sense dictates that private functions should never be called from outside, no matter how desperate the need. (But if you hate rules, even if they are for your own good, see the "Spicey Noodles" section of the Matlab Snack Bar.)
Use the "try/catch/end" mechanism to avoid crashing back to the Matlab prompt when the errors are actually recoverable. To help in that endeavor, the most recent error-message is available as "lasterr". Thus, for example, it is possible to allocate ever larger arrays in a particular process until Matlab fails on an "out-of-memory" error, upon which the chunk-size can be reduced to a somewhat smaller level and the process continued. This sort of scheme could prove to be useful in a combinatorial process.
Dandruff arises most commonly from the use of scripts, as opposed to Matlab functions. Some programmers write sequences of scripts in which the exact names of variables are hardwired into the procedures, and those variables are inevitably planted in the Matlab "base" workspace. Confusion arises easily in such a situation, when any important workspace variable is changed accidentally, or the name of something needs to be changed in several scripts. Also, many of the workspace variables, such as loop-counters, will be irrelevant and unsightly, hence the name "dandruff". By using functions, and by passing cells, structs, and objects between routines, the workspace clutter can be eliminated and the maintainability of the software improved.
Globals (a type of dandruff) are rarely necessary -- I would say never, now that object-oriented programming is available in Matlab. (There is a neat trick that we may have occasion to discuss.)