Writing a large project using J programming language [closed]

Question

Disclosure

This is a "general" question, perhaps without a specific answer, but it is not intended as a flame war. I would really like some information before embarking on my project.

I have to implement a particular project which would really benefit from the data structures and abstractions provided by J. This is a large project, meant to function as the central component of a large (soft real-time) web application. So performance is very important.

I have been trying to find some information about the usage of J in large commercial or open source projects, but I am unable to find any information on which to base my decision to move forward. I have:

• Searched Google Trends, but received the following response: "Your terms - j programming language - do not have enough search volume to show graphs."
• Searched on free(code), and not found a single project using J
• Searched on Sourceforge, and not found a single project using J
• Searched on Lambda the Ultimate, and only found the following discussion that obliquely references APL
• Searched generally on Google and Bing, and failed to find any examples of large scale projects in deployment that use J

Would I be making a mistake in using J for my project? It seems to have everything--especially in terms of data structures, abstraction and concision--that I want. Sure, I could spend time simulating all those properties in F#, or C#, or C++, but J already has them, so...

Can someone please tell me some drawbacks of using J (or any obscure language) for important projects? Is it not sufficiently performant? Does it not have libraries? Anything else I should know?

Thanks in advance for your responses.

Gooseflesh

Probably the main drawback of an obscure language in a large project is in that it is difficult to find many developers with experience in that language. – Ledezma

What key data structures and abstractions present in J are not present in functional languages? – Warram

@DonStewart, to be honest, I would be hard pressed to give you specific examples of data structures, because I know that with a little bit of work all of those can be replicated in Haskell, or any other language. But the hyperdimensional arrays are very useful to me. And the fact that I can manipulate them with a few short verbs means that I can get down to the "meat" of the work, without concerning myself with too much boilerplate code. Of course, all of this is moot if, for instance, J does not have ready XML libraries (and a couple of other things). – Gooseflesh

Concise syntax /= productivity. (($:@(<#[) , (=#[) , $:@(>#[)) ({~ ?@#)) ^: (1<#) – Warram

@DonStewart, I would have to agree with you. To a certain extent, J appears to be a "write-only" language, with lines of code that look like censored curse words. My point was that, once I have a mental lock on the code file, the concision becomes a friend rather than a foe. – Gooseflesh

@DonStewart, I take it that you are tacitly (heh heh) propounding another language? Or at least advising against J? I would take the opinion of someone with your credentials seriously. – Gooseflesh

No, I'm just curious about what structures J has that make it critical for this project. Rarely do decisions swing on a single language feature. – Warram

J libraries for XML can be found as the last three items of this list: jsoftware.com/jwiki/Addons – Disturb

Answer 1

I think J is suitable for critical use on large, important projects. But I also think J would be a very adventurous choice (read: high-risk) for such a project.

The two good-sized companies I'm aware of that use J as the central technology for a major software system are Cognos and Luen Thai. (Cognos has been purchased by IBM.) Some discussion of these systems, or at least some experiences involving them, have shown up in the J Chat Forum. There's not much information there, however.

There may be other big software systems that rely on J, but I've not identified them (and I have been keeping my eye out for years.)

Dmitri Chubarov is right to think that difficulty finding appropriately skilled developers could be a significant problem. Moreover, I think it is difficult for people who have not already developed significant skill in an array language to estimate how much effort it will take them to become good at programming in J. I personally consider it particularly important that J be learned out of an attitude of joyful excitement, and this attitude can be put at risk by the production pressures of a software project. (It is possible to learn J on the job and love it, but the risk that a programmer will hate it because they "have to" learn it should not be ignored.)

Another likely difficulty is integration efforts. J has a variety of libraries, and has been connected to various other software systems, but the contrast with what has been accomplished for more popular languages is dramatic. My experience has indicated that J interfaces pretty readily, which I attribute to some good choices as to how to work with the outside world. (This includes DLLs, memory-mapped files, sockets, and text-file scripts.) Nevertheless, in a large project you'd be innovating much more often than usual when making connection to surrounding support software.

In my mind, the biggest factor would be whether I had an adequate team for undertaking the core product. This might realistically be a much smaller team than most "large" projects rely on. The array languages have been associated with significantly smaller teams than are typical for the field, which I consider to be among the advantages. If you have arranged to get the involvement of whatever colleagues you'll need, great. If not, don't commit to such a project until you do.

The next most important things would be knowing what you expect to obtain as the primary benefits of J, and ways to check that you're getting those benefits as you move through the project. I'm not the right person to advise how to do this, but I am sure you don't want to devote investment capital without keeping track of whether a high-risk choice like this is bearing out the anticipated rewards.

Paul Graham has written that some languages are relied on as secret advantages. J could definitely be such a language. Merely using J, however, would not be enough to assure that it gives the sort of advantages he had in mind. What you described as the appealing aspects of J suggest that such an advantage might be within reach for you. If you do choose to rely on it, it could be vital to have all the major participants have a strong, and sober, understanding why this language was chosen for that work.

Disturb

Thanks for the detailed answer. And bonus points for the link to XML libraries. And yes, "fun" is my starting point. Several parts of the project are still in the planning phase, so I have at least four or five weeks before I need to start writing code for the project. – Gooseflesh

Answer 2

I'm answering this despite the already accepted answer.

• Portability: I'm in charge of maintaining a large code base built with J, most of which is un-portable (or really hazardous to do so) to J6, so I'm stuck in J5. J is not meant to be very backward compatible. In fact, most versions introduce breaking changes. Version 7 uses Java for its frontend, so your interfaces and everything using the Windows Driver (wd) will not work. Version 8 promises to have QT for its frontend, so again your new Java interface will most likely not be ported. Fair warning.

• IDE: To me, large project should go hand-in-hand with a reliable IDE. J's is a text editor.

• Libraries: There are many J libraries, but J being as known as it is, several languages exist which have much richer libraries and frameworks.

• Code structure: That's entirely up to you, and actually not a point against J. J can be written as clearly or as obfuscated as any other language. Standard programming practices apply, and one should always document and describe as explicitly as possible. This counts double in J if you find out that a certain set of "censored swear words" works better than another.

• Finally, if you only need J for some of its capabilities, you can use J as a component. .NET integration has unfortunately been let go in J7 (and wasn't 100% compatible in J6 even, particularly in its 64-bit flavour), but there are ways of making it communicate with the outside world. KDB for example is based on K, J's estranged brother, and basically wraps mapped files for fast access, a facility J has too.

Chromoplast

Thanks a lot for the response. I am very near the point where I have to start writing code. So far, I have been doing more or less parallel unit testing in J and F#. That way, if one of the languages should end up being too troublesome in which to complete the project, I can always fall back to the other. (I'm loving both: the expressiveness of F# and the "nerdiness" of J both have their allure.) – Gooseflesh

That is really a shame that .NET integration has been let go. I always felt that J would be best as an embedded language, i.e., it should be distributed as a library with an optional CLI front-end. Then, you could use J where J is best, and a host language where that is best. – Subplot

Answer 3

For me, J is a good scripting language (in the support role), especially for mathematical related calculations. It is very self-contained to be deployed by folder-copy (especially windows environment), which is advantageous over ruby or python.

For example, in a nodejs webapp, a child_process can be used to call jconsole.exe and pass parameters via command line interface.

The whole program can be deployed and run on windows azure platform, as well as on a local machine without any extra installation (i.e. node.exe and jconsole.exe are included in the same folder).

Bubalo

Answer 4

Can someone please tell me some drawbacks of using J (or any obscure language) for important projects?

The biggest drawback to using more obscure languages for mission critical projects for someone in the role of a manager or project lead are all about people and skills.

1. How do we find and assess potential team member skills and aptitudes in the language?
2. If someone leaves how long will it take me to fill the gaps in knowledge?
3. When we need additional help can we scale the development team fast enough?

For an important project I want people who are already very experienced in the language, and who are not just learning it as they go.

Diaphoretic