Return to Answer

‡ see http://unix.stackexchange.com/tour for guidelines on posts

‡ see https://unix.stackexchange.com/tour for guidelines on posts

Well, okay; I feel it was rather a shame that I spent so much time trying to figure out how to explain my meaning only to find that as I re-read the question. This question ended up "[generating] discussion rather than answers"^‡ and ended up weighing in at ~18K of text (for the question alone, just to be clear) which would be long even for a blog post.

But StackExchange is not your soapbox, and it's not your blog. But you, in effect, have used it as at least bit of both. People ended up spending a lot of time answering your "To-Point-Out" instead of answering people's actual questions. At this point I will be flagging the question as not a good fit for our format, given that the OP has stated explicitly that it wasn't even intended to be a question at all.

_{[original, with clarifications]
have gone far over the time I should spend on this; I apologize for any remaining rough edges}

Just to be clear on what question I'm answering; I was originally going to gather quotes from the question but I think a summary suffices: I may have very well just over-emphasized these, but I think they are relevant. Stated in several different ways, the original post asked questions like why articles would say "Don't parse ls output" or "You should never parse ls output" or similar. [note: not direct quotes].

So, basically my clarification is this: these are all basically a single idiom, but phrased in slightly different ways, stating an imperative together with 'always' or 'never' which, nonetheless are intended as general rules to be used; and when you're a beginner with some subject matter, generally it's best to always follow the general rules without some guidance from someone more experienced; and with rising skill and experience you will be able to, yourself, determine when and if these things are appropriate, and generally you should know when that is because you'll understand all that goes into the problem.

And that's when you choose to do things, perhaps in violation of the "The Rules". But that doesn't make them not general rules. And, it doesn't mean that it would be better to tell a beginner that "sometimes" it's okay to parse ls output, because, it's not. Put your pawns in the center and in the opening one piece, one move, and castle at the earliest opportunity. (Whoops, getting tired, wrong subject.)

General rule general format examples: "You should not ever do X", "Never do X!", "Don't do X!", or "You should always do Y!", and so forth.

In these cases, the general statements are idiomatic uses of phrases that seem to be stating rules valid in all times and places and for all people; but, instead, they're a general rule (or, "rule of thumb", "the basics", etc.) stated in a way that is likely appropriate for the people that read those kind of articles, but those rules are not saying anything about whether an expert might choose to do it when necessary/appropriate. Goto is an example; there's been a big debate on whether they are harmful. (Yeah, don't use gotos. ;D)

To repeat: the feature of a general rule is that they are stated as modal propositions, yet aren't equivalent to the same statements in modal logic (thus, "idiomatic", rather than "literal") and experts break general rules all the time under the right conditions. Instead, it's a way of saying that, unless you have the ability to tell that this statement is incorrect in a particular case, and prove to yourself that you are right, then you should follow the rule.

And this rule is a good one. It's not just that you have to be very good with shell scripting to know whether it can be broken, in some particular case. It's, also, that it's takes just as much skill to tell you got it wrong when you are trying to break it in testing! And, I say confidently that a very large majority likely audience of these articles (that give advice like "Don't parse the output of ls!") can't do those things, and those that do have such skill likely can figure it out on their own.

Just look at this question, and how even people that probably do have the skill thought it was a bad call to do so; and how much effort the author of the question spent just getting to a point of the current best example! I guarantee you on a problem that hard, 99% of the people out there would get it wrong, and with potentially very bad results! Even if the method you've decided on turns out to work; until it (or another) ls parsing idea becomes adopted by IT/developer folk, withstands a lot of testing, especially the test of time, and finally graduates to 'common technique' status, a lot of people might try and get it wrong with disastrous consequences.

So, I do reiterate: especially in this case, that is why "never" is decidedly the right way to phrase it.

_{[UPDATE 2014-05-18: clarified reasoning for answer (above) to respond to a comment from OP; the following addition is in response to the OP's additions to the question from yesterday]}

_{OP's Stated Intention Addressed}

Well, okay; I feel it was rather a shame that I spent so much time trying to figure out how to explain my meaning only to find that as I re-read the question. This question ended up "[generating] discussion rather than answers"^‡ and ended up weighing in at ~18K of text (for the question alone, just to be clear) which would be long even for a blog post.

But StackExchange is not your soapbox, and it's not your blog. However, in effect, you have used it as at least bit of both. People ended up spending a lot of time answering your "To-Point-Out" instead of answering people's actual questions. At this point I will be flagging the question as not a good fit for our format, given that the OP has stated explicitly that it wasn't even intended to be a question at all.

_{As a General Rule...}

_{please forgive remaining rough edges; i having spent far too much time on this already... rather than quote the OP directly (as originally intended) i will try to summarize and paraphrase.}

_{[largely reworked from my original answer]
upon consideration, i believe that i mis-read the emphasis that the OP was placing on the questions i answered; however, the points addressed were brought up, and i have left the answers largely intact as i believe them to be to-the-point and to address issues that i've seen brought up in other contexts as well regarding advice to beginners.}

The original post asked, in several ways, why various articles gave advice such as «Don't parse ls output» or «You should never parse ls output», and so forth.

My suggested resolution to the issue is that instances of this kind of statement are simply examples of an idiom, phrased in slightly different ways, in which an absolute quantifier is paired with an imperative [e.g., «don't [ever] X», «[you should] always Y», «[one should] never Z»] to form statements intended to be used as general rules or guidelines, especially when given to those new to a subject, rather than being intended as absolute truths, the apparent form of those statements notwithstanding.

When you're beginning to learn new subject matter, and unless you have some good understanding of why you might need to do else-wise, it's a good idea to simply follow the accepted general rules without exception—unless under guidance from someone more experienced that yourself. With rising skill and experience you become further able to determine when and if a rule applies in any particular situation. Once you do reach a significant level of experience, you will likely understand the reasoning behind the general rule in the first place, and at that point you can begin to use your judgement as to whether and to what level the reasons behind the rule apply in that situation, and also as to whether there are perhaps overriding concerns.

And that's when an expert, perhaps, might choose to do things in violation of "The Rules". But that wouldn't make them any less "The Rules". 

And, so, to the topic at hand: in my view, just because an expert might be able to violate this rule without getting completely smacked down, i don't see any way that you could justify telling a beginner that "sometimes" it's okay to parse ls output, because: it's not. Or, at least, certainly it's not right for a beginner to do so.

You always put your pawns in the center; in the opening one piece, one move; castle at the earliest opportunity; knights before bishops; a knight on the rim is grim; and always make sure you can see your calculation through to the end! (Whoops, sorry, getting tired, that's for the chess StackExchange.)

_{Rules, Meant to Be Broken?}

When reading an article on a subject that is targeted at, or likely to be read by, beginners, often you will see things like this:

• "You should not ever do X."
• "Never do Q!"
• "Don't do Z."
• "One should always do Y!"
• "C, no matter what."

While these statements certainly seem to be stating absolute and timeless rules, they are not; instead this is a way of stating general rules [a.k.a. "guidelines", "rules of thumb", "the basics", etc.] that is at least arguably one appropriate way to state them for the beginners that might be reading those articles. However, just because they are stated as absolutes, the rules certainly don't bind professionals and experts [who were likely the ones who summarized such rules in the first place, as a way to record and pass on knowledge gained as they dealt with recurring issues in their particular craft.]

Those rules certainly aren't going to reveal how an expert would deal with a complex or nuanced problem, in which, say, those rules conflict with each other; or in which the concerns that led to the rule in the first place simply don't apply. Experts are not afraid to (or should not be afraid to!) simply break rules that they happen to know don't make sense in a particular situation. Experts are constantly dealing with balancing various risks and concerns in their craft, and must frequently use their judgement to choose to break those kind of rules, having to balance various factors and not being able to just rely on a table of rules to follow. Take Goto as an example: there's been a long, recurring, debate on whether they are harmful. (Yeah, don't ever use gotos. ;D)

_{A Modal Proposition}

An odd feature, at least in English, and I imagine in many other languages, of general rules, is that they are stated in the same form as a modal proposition, yet the experts in a field are willing to give a general rule for a situation, all the while knowing that they will break the rule when appropriate. Clearly, therefore, these statements aren't meant to be equivalent to the same statements in modal logic.

This is why i say they must simply be idiomatic. Rather than truly being a "never" or an "always" situation, these rules usually serve to codify general guidelines that tend to be appropriate over a wide range of situations, and that, when beginners follow them blindly, are likely to result in far better results than the beginner choosing to go against them without good reason. Sometimes they codify rules simply leading to substandard results rather than the outright failures accompanying incorrect choices when going against the rules.

So, general rules are not the absolute modal propositions they appear to be on the surface, but instead are a shorthand way of giving the rule with a standard boilerplate implied, something like the following:

unless you have the ability to tell that this guideline is incorrect in a particular case, and prove to yourself that you are right, then ${RULE}

where, of course you could substitute "never parse ls output" in place of ${RULE}. :)

_{Oh Yeah! What About Parsing ls Output?}

Well, so, given all that... i think it's pretty clear that this rule is a good one. First of all, the real rule has to be understood to be idiomatic, as explained above...

But furthermore, it's not just that you have to be very good with shell scripting to know whether it can be broken, in some particular case. It's, also, that it's takes just as much skill to tell you got it wrong when you are trying to break it in testing! And, I say confidently that a very large majority of the likely audience of such articles (giving advice like «Don't parse the output of ls!») can't do those things, and those that do have such skill will likely realize that they figure it out on their own and ignore the rule anyway.

But... just look at this question, and how even people that probably do have the skill thought it was a bad call to do so; and how much effort the author of the question spent just getting to a point of the current best example! I guarantee you on a problem that hard, 99% of the people out there would get it wrong, and with potentially very bad results! Even if the method that is decided on turns out to be a good one; until it (or another) ls parsing idea becomes adopted by IT/developer folk as a whole, withstands a lot of testing (especially the test of time) and, finally, manages to graduate to a 'common technique' status, it's likely that a lot of people might try it, and get it wrong... with disastrous consequences.

So, I will reiterate one last time.... that, especially in this case, that is why "never parse ls output!" is decidedly the right way to phrase it.

_{[UPDATE 2014-05-18: clarified reasoning for answer (above) to respond to a comment from OP; the following addition is in response to the OP's additions to the question from yesterday]}

^{[UPDATE 2014-11-10: added headers and reorganized/refactored content; and also: reformatting, rewording, clarifying, and um... "concise-ifying"... i intended this to simply be a clean-up, though it did turn into a bit of a rework. i had left it in a sorry state, so i mainly tried to give it some order. i did feel it was important to largely leave the first section intact; so only two minor changes there, redundant 'but' removed, and 'that' emphasized.]}

In this case, "never" is an idiomatic use of the word to represent the statement of a general rule; not a statement in modal logic, which, depending on the meaning of "should" in this case may be debatable.

A general rule, however, isn't a statement in modal logic, and frequently general rules can be broken under certain conditions. Instead, it's a way of saying that, unless you have the ability to tell that this statement is incorrect in a particular case, and prove to yourself that you are right, then you should follow the rule.

It's not just that you have to be very good with shell scripting to know whether it can be broken in a particular case. It's also that telling you got it wrong when you try to break it takes just as much. The likely audience of the articles saying such things can't, and those that do have such skill likely can figure it out on their own.

And I think, especially in this case, that is why "never" is decidedly the right way to phrase it.

_{preface and original answer's rationale^†updated on 2015-05-18}

mikeserv (the OP) stated in latest update to his question: "I do consider it a shame though that I first asked this question to point out a source of misinformation, and, unfortunately, the most upvoted answer here is in large part misleading."

Well, okay; I feel it was rather a shame that I spent so much time trying to figure out how to explain my meaning only to find that as I re-read the question. This question ended up "[generating] discussion rather than answers"^‡ and ended up weighing in at ~18K of text (for the question alone, just to be clear) which would be long even for a blog post.

But StackExchange is not your soapbox, and it's not your blog. But you, in effect, have used it as at least bit of both. People ended up spending a lot of time answering your "To-Point-Out" instead of answering people's actual questions. At this point I will be flagging the question as not a good fit for our format, given that the OP has stated explicitly that it wasn't even intended to be a question at all.

At this point I'm not sure whether my answer was to the point, or not; probably not, but it was directed at some of your questions, and maybe it can be a useful answer to someone else; beginners take heart, some of those "do not"s turn into "do sometimes" once you get more experienced. :)

_{[original, with clarifications]
have gone far over the time I should spend on this; I apologize for any remaining rough edges}

Just to be clear on what question I'm answering; I was originally going to gather quotes from the question but I think a summary suffices: I may have very well just over-emphasized these, but I think they are relevant. Stated in several different ways, the original post asked questions like why articles would say "Don't parse ls output" or "You should never parse ls output" or similar. [note: not direct quotes].

So, basically my clarification is this: these are all basically a single idiom, but phrased in slightly different ways, stating an imperative together with 'always' or 'never' which, nonetheless are intended as general rules to be used; and when you're a beginner with some subject matter, generally it's best to always follow the general rules without some guidance from someone more experienced; and with rising skill and experience you will be able to, yourself, determine when and if these things are appropriate, and generally you should know when that is because you'll understand all that goes into the problem.

And that's when you choose to do things, perhaps in violation of the "The Rules". But that doesn't make them not general rules. And, it doesn't mean that it would be better to tell a beginner that "sometimes" it's okay to parse ls output, because, it's not. Put your pawns in the center and in the opening one piece, one move, and castle at the earliest opportunity. (Whoops, getting tired, wrong subject.)

General rule general format examples: "You should not ever do X", "Never do X!", "Don't do X!", or "You should always do Y!", and so forth.

In these cases, the general statements are idiomatic uses of phrases that seem to be stating rules valid in all times and places and for all people; but, instead, they're a general rule (or, "rule of thumb", "the basics", etc.) stated in a way that is likely appropriate for the people that read those kind of articles, but those rules are not saying anything about whether an expert might choose to do it when necessary/appropriate. Goto is an example; there's been a big debate on whether they are harmful. (Yeah, don't use gotos. ;D)

To repeat: the feature of a general rule is that they are stated as modal propositions, yet aren't equivalent to the same statements in modal logic (thus, "idiomatic", rather than "literal") and experts break general rules all the time under the right conditions. Instead, it's a way of saying that, unless you have the ability to tell that this statement is incorrect in a particular case, and prove to yourself that you are right, then you should follow the rule.

And this rule is a good one. It's not just that you have to be very good with shell scripting to know whether it can be broken, in some particular case. It's, also, that it's takes just as much skill to tell you got it wrong when you are trying to break it in testing! And, I say confidently that a very large majority likely audience of these articles (that give advice like "Don't parse the output of ls!") can't do those things, and those that do have such skill likely can figure it out on their own.

Just look at this question, and how even people that probably do have the skill thought it was a bad call to do so; and how much effort the author of the question spent just getting to a point of the current best example! I guarantee you on a problem that hard, 99% of the people out there would get it wrong, and with potentially very bad results! Even if the method you've decided on turns out to work; until it (or another) ls parsing idea becomes adopted by IT/developer folk, withstands a lot of testing, especially the test of time, and finally graduates to 'common technique' status, a lot of people might try and get it wrong with disastrous consequences.

So, I do reiterate: especially in this case, that is why "never" is decidedly the right way to phrase it.

_{[UPDATE 2014-05-18: clarified reasoning for answer (above) to respond to a comment from OP; the following addition is in response to the OP's additions to the question from yesterday]}

† I originally intended this solely as a clarification on my original; but decided on other additions upon reflection

‡ see http://unix.stackexchange.com/tour for guidelines on posts