I think the meaning of this sentence is not clear enough, but it should not be considered obviously bogus. It does not mean that ONLY covariant interfaces can inherit non-generic interfaces.
I have done some archaeological work, and the earliest version of this sentence I can find is on Jan 30, 2006. The original text is:
Generic interfaces can inherit from non-generic interfaces if the generic interface is contra-variant, which means it only uses its type parameter as a return value. In the .NET Framework class library, IEnumerable<T> inherits from IEnumerable because IEnumerable<T> only uses T in the return value of GetEnumerator and in the Current property getter.
It can be seen that there have been no changes except for the "contra-variant" fix. The author of this sentence is most likely Brad Abrams, as the same mistake appeared in his blog post 'Why does IEnumerable inherits from IEnumerable':
As it turns out, the only generic interface for which this is possible is IEnumerable<T>, because only IEnumerable<T> is contra-variant: In IEnumerable<T>, the type parameter T is used only in "output" positions (return values) and not in "input" positions (parameters).
Returning to the sentence, it is obvious that we know regardless of whether an interface is covariant, contravariant, or invariant, it can inherit from another interface.
However, let's focus on the context, the previous sentence is:
The rules of inheritance that apply to classes also apply to interfaces:
So this paragraph first clarifies the general interface inheritance rules, and then describes a specific inheritance rule. Therefore, the problematic statement does not negate other inheritance rules in the C# language.
The problematic statement is about the relationship between IEnumerable<T> and IEnumerable. I think we should not consider generic interfaces and non-generic interfaces as arbitrary interfaces. The definition of non-generic interfaces here should be the counterpart of generic interfaces, that is, I <-> I<object>. This is also mentioned in the post above:
Ideally all of the generic collection interfaces (e.g. ICollection<T>, IList<T>) would inherit from their non-generic counterparts such that generic interface instances could be used both with generic and non-generic code.
Also we should not consider "the generic interface is covariant" as a necessary condition for inheritance. Please pay attention to the following sentence in the same article:
Generic classes can implement generic interfaces or closed constructed interfaces as long as the class parameter list supplies all arguments required by the interface
There is a difference in the verbalism compared to the problematic statement in terms of relevance. In other words, the sentence can be expressed as: Generic interfaces are not required to inherit from their non-generic counterparts if the generic interface is not covariant.
Regarding this point, Eric Lippert explained as follows:
BONUS QUESTION: Why does IEnumerable<T> inherit from IEnumerable but IList<T> does not inherit from IList?
A sequence of integers can be treated as a sequence of objects, by boxing every integer as it comes out of the sequence. But a read-write list of integers cannot be treated as a read-write list of objects, because you can put a string into a read-write list of objects. An IList<T> is not required to fulfill the whole contract of IList, so it does not inherit from it.
Finally, I checked that all covariant interfaces in the .NET library (IEnumerable, IEnumerator, IQueryable, IOrderedQueryable) inherit their non-generic counterparts, while none of the non-covariant interfaces do so. So I think describes is not a language specification, but a design rule.