That particular help topic is poorly written; the "Tip" box seems to suggest that you can use those sort of formulas, but you can't even use a formula that includes a field even on the current object. The tip is actually trying to say that you should use a cross-object formula instead of a default value if you want to show data from a related record. This has to do with the fact that default values are static and unchanging, while cross-object formulas can change to reflect changes in data after the record is created.

Default values are a way to provide a value so that the user does not necessary need to select a value. The only values that are available are those that are available when the new record page is rendered initially. The only values that are available, then, are global variables, which is why they appear in the list of fields you may insert. Cross-object values would not be available, because the user would have to populate the lookup field before a default could be derived, but defaults are derived only when the page initially renders. Default values would be a lot more useful if they could be dynamic while the user was editing the new record instead of during initial rendering, but that is not yet a possibility (at least, not without Visualforce).

Hopefully I clarified things for you.