The reason TeX is still used is because it is open source and beautiful, because it is the best at handling mathematical notation, and because of its inertial dominance in math and the hard sciences in Academia. The reason TeX has so, so many fixable problems after 30 years is because there is no financial incentive for anyone to fix it.
The answer is not to find a TeX alternative. This will not be possible in Academia for a very long time. The answer is for someone to plunk down a million dollars to hire some full-time developers to fix it. Honestly, the TeX language is wonderful; it's the editors which are terrible. A TeX editor should...
[] seamlessly hide the compilation process
[] feature a modern debugging environment
[] make 99% of commands (including everything you can do in MS Word) discoverable within the editor, rather than sending the user to the internet or some pdf guide, and
[] robustly handle all modern image types
...without becoming an unstable WYSIWYG hybrid like LyX. The only thing about the language itself that might need to be changed is some standardization of common packages. I shouldn't need to download anything to get appendices and sub-figures to behave sensibly.
The amount of professor, post-doc, and grad student time wasted on TeX is staggering, and easily justifies paying for an overhaul...if only the government or some benefactor would step up.
I would disagree that the base is wonderful and it's the editors that are bad. At least in LaTeX there are a multitude of packages to do any given thing, all with varying levels of completeness, and most with incompatibilities between them. Leaky abstractions abound, and the way the macro language works probably makes this easier.
The features you mention would be needed to put it in front of less geeky users but are far from the core issues. I'd love to see a language with better flow control and abstraction features put on top of the Tex engine that indeed produces beautiful output. Maybe LuaTex will be that.
The core algorithms are pretty creaky as well. Knuth did a brilliant job coming up with efficient algorithms that could do a fairly good job at breaking lines/paragraphs/pages and could handle book-length documents even on 70s-era hardware, but they've had minimal improvement since then. Figure placement in particular is pretty bad, especially for two-column, relatively short conference papers, which is currently a frequent TeX use case. And all of those things are really hard to exercise any control over, with black-magic parameters that often don't have obvious effects ("goddammit, why won't it put the figure at the top of page 5?!").
LuaTeX does seem to be experimenting with some improvements in these areas though.
All true. Also, the way indexes, table of contents, bibliographies and cross references work in LaTeX are unholy, fragile hacks. Foot- and endnotes could use some TLC. And on 2011-era hardware we really should have some approximation of globally optimal page breaking.
Ugh, and fonts and character sets. Do you still have to put in magic incantations like
to get working hyphenation with wacky characters such as æ, ø, and å?
LuaTeX does seem to be experimenting with some improvements in these areas though.
About when I stopped paying attention to TeX and LaTeX many years ago (2002-ish?), there was some effort under way to rewrite all of TeX in Java. Did that go anywhere?
> About when I stopped paying attention to TeX and LaTeX many years ago (2002-ish?), there was some effort under way to rewrite all of TeX in Java. Did that go anywhere?
I don't have any insight at that level of abstraction. Figure placement like a bunch of other things has required me to hack around to get my desired result. Is there any newer research on these algorithms that could make it better given much better hardware?
As for LuaTex I wasn't able to find any examples of actual markup written in it. I would expect that for the actual markup you'd want a language with a bigger and more flexible grammar so you could build DSLs. Ruby seems particularly good at that kind of stuff, but I'm biased there.
LuaTeX has the same macro language as the other TeX engines (pdftex and it's ilk). The true power of LuaTeX comes into play when you use Lua scripting. I am not sure if ConTeXt has more advanced figure placement algorithms due to LuaTeX. LaTeX for sure has the same algorithms as before.
But then is the Lua scripting at the user level? Can it be used in the building of documents? Or is it a scripting language for the underlying engine, invisible to the end-user?
Incidentally I went looking for what pure TeX actually looks like as most people say TeX when they really mean LaTeX. Here's Knuth's TAOCP Volume I errata:
I wrote a LaTeX macro package for my university's PhD thesis format, and like a lot of such packages, the bulk of the work was in TeX rather than LaTeX. It's...well, I'm struggling to find the appropriate word that means "awful, but sort of comfortably awful."
Like a lot of things, if you're handed a finished product, it looks like random noise. If you could watch it being written, it pretty much makes perfect sense. It's a hideous syntax, and sometimes I still wake up in a cold sweat with long strings of '#' signs running taunting me from the corner of my eyes. But I can't say it was all that daunting to write, although I suspect my TeX code was highly non-idiomatic.
You can do Lua scripting a the user level (\directlua{...}). But you have to know the Lua interface. What purpose is having Lua inside your document? Do some calculations? Yes, that's easy. Do complex algorithms for placing floats? Then you need to replace a lot of TeX/LaTeX code anyway and then I wouldn't speak of user level. What you can do easily from inside Lua: shuffle nodes (nodelists) around. Everything you see on a page is represented by a node. So you can reposition anything. That requires a lot of knowledge of the internal representation of nodes and the LuaTeX API, but once you grok it, it is much fun to do these things.
And no, Knuth is definitely not the only one using plain TeX. Yes, it is a niche.
I did a few times. It's actually easier to make something look exactly the way you want it to look in plain TeX than fight with some poorly documented LaTeX package.
I'm sure its possible and useful for specialty cases but would you want to write TAOCP in it?
My experience of using LaTeX compared to WYSIWYG is that it's easier to focus on the text in the editor instead of fighting with the UI interface but it then requires some chasing around package bugs and compile errors. In total it's clearly a net positive for me.
Writing pure TeX would seem to push the tradeoff quite a bit, having so much distracting stuff around the text that requires a totally different mindset than the text being written. It was probably designed for a world where typesetting was done as a final step before publication, potentially by someone else.
The idea is that you put together the few macros that you need and then the book is mostly tag-free.
IDK you can find the source to the TeXbook on CTAN, see if it seems too noisy to you (I don't remember).
>I would disagree that the base is wonderful and it's the editors that are bad
OK. (As I said elsewhere...) When I said the language was fine, I meant that the language as a means for the user expressing an idea was fine, not that there aren't problems at a deeper level with the language. I'm a physicists, not a programmer. Nevertheless, the limitations of the given language could be massively ameliorated with a powerful editor.
>At least in LaTeX...
Sorry, I wasn't clear. I wasn't distinguishing between TeX and LaTeX. Everyone I know in Academia used LaTeX. I was just using TeX as a catch all to distinguish from other typesetting options.
> The features you mention would be needed to put it in front of less geeky users...
There's no reason I should have to go on the internet to find out how to use columns. Or sub figures. Or appendices. It should just work.
I'm not a less-geeky user who needs to be coddled (unless by "less geeky" you mean 99.99% of users). I just want to write up my research, and I don't want to have to think about typesetting theory.
>There's no reason I should have to go on the internet to find out how to use columns. Or sub figures. Or appendices. It should just work.
>I'm not a less-geeky user who needs to be coddled (unless by "less geeky" you mean 99.99% of users). I just want to write up my research, and I don't want to have to think about typesetting theory.
I agree that those things should be easy to find. What I meant is that for the current user base those are nice to have but not essential. I have long since accumulated enough examples of the type of documents I write that I can quickly do the basic things just by copying them from somewhere else. I assume that people that write scientific articles already have working examples of two columns, sub figures and appendices that they just copy when they do a new article.
If you wanted to make LaTeX much more widely use then you'd need all the goodies you mentioned. But to even make those possible the environment under them needs to be much more consistent and robust than it is now. Something like LyX could work well if manual intervention in the typesetting wasn't so needed.
I guess we're disagreeing about what's essential. At this point, I prize robustness more than power. I'd be pretty happy if LaTeX would just work smoothly. Additional capabilities would be gravy.
I'm confused. You're the one that wanted extra features in the environment. All I hoped for would be for the typesetting engine to be more robust and modules to work well together. All I want are bugfixes.
To add to your fourth point, it should also ideally handle all modern font types and all of their provided features in some relatively simple way. XeTeX/XeLaTeX is great, but that it can’t always do the right thing when any regular TeX document is dropped into it is unfortunate (cf. http://scripts.sil.org/cms/scripts/page.php?item_id=xetex_fa...), and it doesn’t support everything it would in a perfect world.
There are several editors for Tex&co that do on-the-fly compilation. Have you seen Kile's two panelled interface, for example?
feature a modern debugging environment
Well, this is a problem with the language: macro expansion is notoriously tricky to debug. I'd like to see work done on a static analyser for Tex.
robustly handle all modern image types
This is not a problem I have ever had. What are you thinking of? The only file types needing special handling in Luatex are Postscript and SVG; both convert painlessly to PDF, and the converter can be run automatically by Luatex.
> There are several editors for Tex&co that do on-the-fly compilation...
Yes, I have two-panelled TeXworks open as we type. It's not seamless because I had to spend an hour fussing with it to set it up with my LaTeX distribution, and another 2 hours dealing with it when I needed to move from pdflatex to regular latex because the journal I was submitting to used some bizarre packages.
It's not that you can't eventually get it working, it's that you ever have to think about it at all.
>Well, this is a problem with the language: macro expansion is notoriously tricky to debug. I'd like to see work done on a static analyser for Tex.
Fine. When I said that the language was fine, I meant that the language as a means for the user expressing an idea was fine, not that there aren't problems at a deeper level with the language. I'm a physicists, not a programmer. Still, I can't imagine the debugging situation can't be greatly improved with the given language if actual people were employed to fix it.
>This is not a problem I have ever had. What are you thinking of? The only file types needing special handling in Luatex are Postscript and SVG; both convert painlessly to PDF, and the converter can be run automatically by Luatex.
I have had many, many headaches with getting images into a document. Last month, I spent 4 hours trying to solve one particular problem. Yes, maybe if I were deeply familiar with all the technical stuff surrounding TeX's relationship with images, I could have figured it out.
It should just work. I should type \includeimage{filename}, and it should just work.
I'm glad you like LyX, and I encourage other people who haven't found a TeX editor they like to try it out.
Personally, I don't want my editor to do anything with my source code except color code it, like turning the source code into into manipulable objects (or whatever you call that).
> LyX is a document processor that encourages an approach to writing based on the structure of your documents (WYSIWYM) and not simply their appearance (WYSIWYG).
Their written description notwithstanding, LyX is a hybrid between WYM and WYG. It bundles up standard blocks of source code and hides it behind graphical objects which resemble the final product.
I believe that description is aimed at former MS Word users who don't even know the WYM/G distinction exists, not as a definitive membership claim by LyX.
TeXmacs is a true WYSIWYG scientific word processor. It's not a TeX/LaTeX GUI front-end. Rather, it has its own typesetter that produces quality comparable to TeX in real-time as you type.
Honest question: What makes you say that? I've never completely understood why TeX is considered the best at handling mathematical notation. Personally, I've been very happy with the equation editor in Word 2010. I'll certainly acknowledge that there's likely better solutions than Word 2010, but before I invest substantial time learning something like TeX I'd like to understand exactly what it is I'm missing out on.
Do you typeset any serious mathematical work in Word 2010?
When people talk about TeX's mathematical typesetting abilities, they are often (from personal experience, as a math major currently in school) referring to having to typeset documents with many pages and upwards of 25% being math. It's much easier for me to type \int_0^\pi \sin (x) dx than it is for me to go click-click-click-click in Word. But besides that, when you're writing proofs, for example, and you have to keep referring to the same symbol, say sigma, over and over again, in LaTeX, you would simply write $\sigma$, but in Word, you would have to do Insert->Symbol or some nonsense. Several more seconds wasted, and that adds up when you have 100 occurrences of the character sigma in your document.
TeX has a learning curve, but it's not even actually that steep. Within 4 days of starting to use it I was already starting to grok it.
"... The answer is for someone to plunk down a million dollars to hire some full-time developers to fix it. Honestly, the TeX language is wonderful; it's the editors which are terrible. A TeX editor should... ..."
Would it really take a $1M dollars to achieve a useful enough fix for TeX editing? And a better question, "is there are market for such an editor?"
Yes. Read Yossi Gil's answer: http://tex.stackexchange.com/questions/13370/13435#13435. Here's an excerpt:
I believe most users (maybe not so in this site) just turn a blind eye to the not so elegant features, and produce amazing documents in no time. The very friendly bunch here does not see these not-so-elegant portions as being so annoying. To most, these are just riddles to enjoy, just like cryptic crossword puzzle.
....
The difficult part is in programming something new, something that cannot be found in existing package. There are experts of this dark art: the rest of us just watch them do their wizardry with awe. Luckily, the cases where you really need to do this kind of programming are very rare: chances are that you will find it was done.
In my experience, this is the state of the language. If you could fix this, I believe that even those who currently enjoy the riddles would move over.
That's probably because Linux has more utility for its audience than TeX does. Which is sad, as TeX's original goal was to provide typesetting for the masses, who are unlikely to ever use it for their typesetting purposes.
Nope, TeX's original (and only) goal was to help Knuth typesetting his books (TAOCP specifically) and he didn't think anyone but his secretary will use it.
Replying to someone else's statement with 'nope' and your own different understanding of facts is inane and rude. Either ask for a citation - which I'll gladly provide - or provide a referenced counterpoint of your own.
Assuming you're actually interested in some kind of discussion rather than just trolling, see http://papers.ssrn.com/sol3/papers.cfm?abstract_id=908946, the same reference that's provided on the Wikipedia page for TeX if you'd bothered wondering where I got the information from, rather than just replying 'no' without any research.
The quote is: "TeX was designed with two main goals in mind: to allow anybody to produce high-quality books using a reasonable amount of effort, and to provide a system that would give exactly the same results on all computers, now and in the future."
A simple Google search for the statement above will reveal it's in almost every document ever produced on the topic of TeX, which suggests it may have been written by Donald Knuth himself.
Regardless, TeX failed for the masses because it was too complicated to begin with (the masses are not programmers, nor are they interested in "programming" their documents). WYSIWYG word processors and desktop publishing software do a decent enough job quickly with a learning curve orders of magnitude shallower, and rightly so since it's the learning curve that poses the greatest barrier to entry.
And while many attempts have been made to flatten that curve with various tools, I'm hard pressed to find a compelling competitor to Microsoft Word or OpenOffice in terms of ease-of-use.
People in general are far more interested in the final result than the "beauty" of the steps to produce that result, especially when it comes to the written word.
And still most majority of kernel hacking is done by private people in their free time. No mention of virtually every other software I use everyday. Think of drivers, often made and improved AGAINST a company will, by reverse engineering closed sources.
Money have nothing to do with open source. Sure they help, but if a project is not followed it's not because of money incentives.
That might have been the case 10 years ago but not today. According to G. Kroah-Hartmann, more than 80% of kernel contributors are full-time paid developers, the remaining 20% being spread among free-time contributors and companies who do not want to disclose their names. People contributing to gcc, mozilla or OpenOffice are all full-time paid developers funded by a company to do so.
Money is not everything, but it has a lot to do with everything, including open source. Linux would still have been a hobby operating system if a number of big players (Sun, IBM, etc) didn't get on the Linux bandwagon in the early 2000s.
I use scribtex.com. It handles all the packages and compilation process. I can't say, really, about the debugging, and I haven't used different image types, but moving Tex on Latex online fixed a lot of problems for me.
The answer is not to find a TeX alternative. This will not be possible in Academia for a very long time. The answer is for someone to plunk down a million dollars to hire some full-time developers to fix it. Honestly, the TeX language is wonderful; it's the editors which are terrible. A TeX editor should...
[] seamlessly hide the compilation process
[] feature a modern debugging environment
[] make 99% of commands (including everything you can do in MS Word) discoverable within the editor, rather than sending the user to the internet or some pdf guide, and
[] robustly handle all modern image types
...without becoming an unstable WYSIWYG hybrid like LyX. The only thing about the language itself that might need to be changed is some standardization of common packages. I shouldn't need to download anything to get appendices and sub-figures to behave sensibly.
The amount of professor, post-doc, and grad student time wasted on TeX is staggering, and easily justifies paying for an overhaul...if only the government or some benefactor would step up.