Is it time for a new Forum?

REAS

What do you think? This Forum works, but can it work better?

This forum has been running since 2003, that's fifteen years! In that time, we've rebooted it three times and we've used three different software systems to talk with each other. I think it might be time to move to a new system and to start again from scratch, but I think it's important to discuss and to try to reach a consensus.

The idea on the table at this moment is to start a New Forum on Discourse and to encompass all of the Processing Foundation software: Processing, p5.js, Processing for Android, Processing.py, Processing for PI, and more!

billautomata

Yes. A discourse / stackexchange style server would modernize this forum greatly.

kfrajer

Not sure if it is a good idea. I mean, what features are missing? [Yes... this needs to be discussed]

What features do you want to get from stackoverflow? Do you want to bring the voting system? Please, don't... I don't find it useful in the long run.

You can add features to the current forum, like copying code block by single click, improve search engine including applying filters, allow to underline text (similar to bold and delete text) when writing a post. There is a need to fix few bugs: @ Overload, http//: www.link.com, //*****, any others?

There is lots of repeated efforts done by volunteers working in the same topics over the many months or many years. It is really hard to reference those topics and most of the time you end up writing the same code with small variance. I guess the best thing is to have the posts properly categorized and to have a better tag system. The tag system I envision is something similar to what one finds in the issue tracking in github: bug, improvement, novelty, help wanted, etc. Well, there is also tags related to the content of the post: openGL, p5.js, visual effects, arduino, etc. which should be used by the search engine.

Do we need a new forum? Will you make this a GSOC project?

Kf

hamoid

I vote for Discourse :) I've used the OpenFrameworks forum for the last two years and I'm very happy with it.

I started writing a long reply with many details, but I'll summarize instead: with Discourse I can find help faster and help others better thanks to its good design, amazing search and real time functionality. You can try it here: https://try.discourse.org/

@kfrajer I don't think Discourse has any voting system.

kasperkamperman

I think Discourse is a great option and a good switch. More modern and responsive (try altering the width of this forum). I'm more active on the Particle discourse forum nowadays (also due personal interests) and what nice is that you can watch conversations. Also e-mail alerts and everything seem more user-friendly, or at least they look nicer.

KevinWorkman

tl;dr: Discourse++

I think the forum is definitely a place we can make a lot of improvements. It's the first place a lot of novices go when they're stuck, and if we get it right we can help a lot of people stick with it and achieve more of what Processing is all about.

I think it would be useful to talk about what problems we're trying to solve. The way I see it, there are two main issues:

1. The forum software itself. Making it easier to format code, making it easier to edit posts, making moderation more effective. The forum software isn't exactly terrible, but it's not very engaging or fun to use like Discourse is. The fact that it's harder to use encourages a lot of bad behavior.
2. The forum culture. Honestly this is a bigger deal to me than the forum software.

We get a lot of people asking questions that are simply not very answerable. They're either too broad (how do I do X) or include 500 lines of code that haven't been debugged. I love helping novices, and I think we can steer these folks to help them ask better questions and be more comfortable doing their own research. But right now there isn't a clear policy or set of references or anything, so it's like starting from scratch with every new post.

Similarly, some of the answers can tend to be code dumps instead of actually helping people solve their problems. These posts might help the OP in the short-term, but I think it does a huge disservice to both the OP and the forum in general. I think we need a policy that encourages actually helping instead of doing people's homework for them.

The idea on the table at this moment is to start a New Forum on Discourse and to encompass all of the Processing Foundation software

I would love this. I'm a huge fan of Discourse. I think this solves all of the problems with point 1 above, and goes a long way towards solving the culture problem as well.

So to summarize, I think we should do three things:

1. Switch to Discourse. Mothball the existing forum and start from scratch. I'd also go so far as to say we should cut back on the number of moderators, because Discourse has built-in community moderation tools.
2. Establish a set of policy pages. Is it okay to do homework for people? Is it okay to dump full code solutions without any explanation? Enforce these by banning users who continually violate them.
3. Establish a set of helpful guides. How to break your problem down into smaller pieces. How to create a MCVE or SSCCE. How to debug your code. How to use your developer tools. The difference in the various Processing versions. These are just a few examples of the information that we seem to give over and over again (that's why I wrote these guides in the first place), and having one place for all of it would be very helpful.

Chrisir

Problems I see for beginners:

• no automatic code recognition and formatting

• Not possible to upload entire project with images and csv/ mp3 files so it’s often impossible to run the code

I would advise against unifying the existing forum and instead mothball this and replace this forum first and keep them separate first

Chrisir

jeremydouglass

Code: A huge amount of energy goes into teaching each newcomer to post code -- so a platform change could be helpful there.

Search: Also, search on Vanilla is terrible, with regulars recommending Google search instead -- so again, a platform change could be very helpful. Answers are given, but people can't find them, and that creates a lot of duplication of effort.

Reuse: While I think the forum should stay learner-centered and project-centered, rather than prioritizing becoming an archive of authoritative answers, it would be nice to have more reuse of standard questions / answers -- an SO-like method of dealing with the 300th time someone asks "How do I remove something while iterating over from an ArrayList?"

Old Dates: Mothballing all old content every few years is also part of the problem, not just a solution -- many questions get answered over and over because old but perfectly good answers keep getting shifted into read-only zones, and then nobody can put an addendum on them or update them or write "no longer true!" at the bottom. When the answers are not so good, one of the biggest problem with previous mothballings is that not only can we not update them, they have baked in relative timedate stamps. There are 5-10 year old questions on the old static forums that still say "1 year ago" on them -- permanently -- and that can be seriously misleading as it makes advice seem recent and relevant that isn't. It would be nice if before mothballing this forum we could avoid making that specific mistake again.

GoToLoop

This Forum works, but can it work better?

Well, the previous Zoho forum was much better than this Vanilla 1 IMO: $-)
https://Forum.Processing.org/one/

It was more colorful & fun; and even had a leaderboard. <:-P
And most importantly, no butchered code posts! \m/

I know it was dumped b/c of weak SPAM protection. But this 1 is dull & unfun. :-\"

The main issue w/ this forum, besides being unfun, is its markdown system, which butches every single code post here since the very 1st day! :-O

More specifically, the dumb syntax that we need to end a sentence w/ at least 2 trailing spaces in order to have a new line break! [-(

It's interesting the fact that even though GitHub uses markdown as its posting system, that 2 trailing spaces syntax isn't needed there! @-)

Also, they accept the triple ``` fenced block syntax, which allows us to post code w/o 4-space indentation; plus specify a language for syntax highlight! :>

Seems like folks here wanna move onto Discourse. I didn't know that 1. But it seems cool. B-)

However, it says it uses a markdown variant called CommonMark.
And again, it's got that 2 trailing space syntax just like this forum follows: :o3
http://Spec.CommonMark.org/dingus/?text=foo baz

In other words, moving onto Discourse won't solve the main problem our forum users actually need:
An easy & straight code copy & paste from PDE into the forum here! 3:-O

KevinWorkman

@GoToLoop: If you've never used Discourse before, you can play in their sandboax at https://try.discourse.org. It gets reset every 24 hours so you can feel free to test formatting and stuff.

I don't know what you mean by the trailing spaces thing. I just use the enter key to separate my paragraphs. Honestly I find a bunch of short paragraphs pretty hard to read, but YMMV.

Anyway I tried various code formatting techniques on the Discourse sandbox, and it looks like this:

If you just paste, newlines are preserved, but indentation is not. You can also use three tick marks to fence code, or the forum has a code button which indents your whole thing 4 spaces to indicate that it's a code block.

Another nice thing about Discourse is that it's much easier to edit posts. In our current forum, I believe the indentation is stripped when the user posts, so we have to get them to repost and format their own code, which as you know is a losing battle. In Discourse, the indentation is preserved (just not shown in the rendered markdown), so it's as easy as clicking the pencil icon (which edits directly in the page instead of loading a separate edit page) and adding the code fences. The user will then get a notification that their post has been edited, so they'll see the proper way to format and hopefully learn from that. No more "can you please repost your code and this time click the edit button" which is honestly like 75% of our job here.

GoToLoop

Still no account there, so I can't test-post yet.

But if Discourse respects newlines (like GitHub) w/o demanding 2 trailing spaces at the end (like here and Reddit), so it's a solution for our headaches here! >:D<

So it turns out Discourse doesn't follow strict CommonMark syntax like I thought: ^#(^
http://Spec.CommonMark.org/0.28/

hamoid

I strongly agree with point 3 @KevinWorkman : a better forum would definitely help, but it should go together with tutorials and hints to help people ask better questions. I recently opened this issue in Github which I can paste here:

Idea: remind users how to ask good questions

Hi! I see many questions in the forum which do not provide enough information to be able to help. Something like "X is not working. I tried many things. What can I do?". So the first step would be to write back and ask for those details that should have been there in the first place.

I think it might help to include a text above the form in the forum explaining in a few words how to ask good questions to get an answer faster (tell what you did, what you got, what you expected, what are you trying to do, show code, maybe include OS and P5 version, etc). Or even link to something like https://stackoverflow.com/help/mcve

This is how it looks currently:

Maybe a "tips for getting help faster: bla bla ..."?

---

@GoToLoop: Discourse shows side by side the markdown and the rendered result in real time. No need to click "preview". This makes it much easier to make sure your post looks right and should solve issues with line breaks, code formatting, etc.

ps. What do you think about collectively writing a page about "how to get help faster", showing a list of issues found in questions (not enough info, too complex example, improper tone, inefficient communication, etc.) and the solution to each issue, trying to make the person understand that there's a volunteer in the other side, with feelings and limited time, etc. Basically, "try to make it easy for others to answer your question".

KevinWorkman

@GoToLoop: It's a sandbox. You can register for an account for free. It takes about 30 seconds. And I still don't know what you mean by the trailing spaces thing. Do you mean with code blocks?

@hamoid: Another nice thing about Discourse is that it pretty much has this feature out of the box. It shows a message to new posters, and also recommends looking at posts that look similar to what's being typed.

GoToLoop

And I still don't know what you mean by the trailing spaces thing.

Gonna repeat the links about the subject:

1. http://Spec.CommonMark.org/dingus/?text=foo baz
2. http://Spec.CommonMark.org/0.28/#hard-line-breaks

KevinWorkman

@GoToLoop: Can you please explain why this is an issue for you? I'm trying to understand your use case. Linking to a spec doesn't really explain why it's a problem for you.

koogs

Making it easier to format code

it's currently one keypress. the problem is that people just post code without formatting it and the forum is (understandably) too dumb to do it itself - how does it know? is any other forum going to be any better at this?

i'd like a forum without animated emojis.

GoToLoop

... and the forum is (understandably) too dumb to do it itself...

The forum doesn't need to recognize whether or not some part of a post is actually a code block or a normal block. What it mustn't do is butcher the post!

I've already mentioned that; but here it goes again:
GitHub does the right thing w/ its own markdown version.

If n1 simply copy his/her sketch from the PDE, and paste it as a GitHub comment w/o any modification, that post won't be recognized as a code block.

However, it won't get butchered as it is the case here or even at Reddit!

Why you ask: B/c GitHub's markdown doesn't demand at least 2 extra spaces at the end of a line in order to get a newline break!

So I'm gonna ask again: What about Discourse?

Does a pasted block code gets butchered like it is the case here or it is treated as a normal block text like at GitHub, respecting its newline breaks? L-)

KevinWorkman

i'd like a forum without animated emojis.

Hahaha. This + 1000. Discourse emojis are not animated.

KevinWorkman

So I'm gonna ask again: What about Discourse?

Does a pasted block code gets butchered like it is the case here or it is treated as a normal block text like at GitHub, respecting its newline breaks?

Please see my above screenshot, which includes some code that I just pasted in without adding any formatting. It preserves newlines, but does not preserve indentation. But editing in Discourse is much easier, so adding the code fences is much simpler.

GoToLoop

That's exactly what happens at GitHub. We've got newlines but no indentation. :-B

It's not 100% ideal, but it's much bearable to read if some1 still doesn't know how to properly post code blocks. 8-X

Which btW, it's the behavior of the majority of folks who come for our help here! :-\"

quark

I have been playing with the Discourse sandbox and it appears the Preformatted Text button behaves the same way as the C button on this forum. (It indents the code block 4 characters and you still need a blank line either side of the block. It also preserves the indentation).

The advantage is the live preview which makes it very clear what the final post is going to look like. This feature is fantastic for newbies (and me) who can experiment getting the markdown correct BEFORE posting live.

Obviously using the code fence is better because it does give some syntax highlighting.

It might be possible to customise the Discourse's edit box to add other buttons e.g. code fence.

+1001 It will be good to get rid of the animated emojis!!!

So far I have been very impressed with Discourse

All we need then are better formulated questions followed up with informative answers

Chrisir

Well, my points have not drawn too much attention

Anyway, I don’t think we can solve the content and quality issues wlth a new technology. Two different things unfortunately.

I think we can work on the quality of our answers, but not so much on the quality of the questions. I think it should be the privilege of newbies to ask questions falsely. I remember finding even the links on the processing main page confusing. So it's great when they manage to find the forum, register and ask a question.

• Remember: For some of the newbies it’s there first programming experience and there first time in a forum (any forum). So I think we want to be patient.

quark

It is not just newbies that ask poorly formulated questions, we need patience when responding to anyone. ;)

I think KevinWorkman has covered all the major points in his first post above, especially his comments on forum culture.

I think we can work on the quality of our answers

Very true, that applies to everyone using the forum, including me.

KevinWorkman

Well, my points have not drawn too much attention

Sorry, which points would you like to have addressed?

Anyway, I don’t think we can solve the content and quality issues wlth a new technology. Two different things unfortunately. I think we can work on the quality of our answers, but not so much on the quality of the questions.

I don't think this is necessarily the case. One of the core principles of Discourse is to use "just in time" prompts to improve user behavior. Here is an interesting article discussing it. But basically, little stuff like popping up a message saying "Don't forget to format your code" or "Please narrow your problem down to a small example program" right as the user is writing their post goes a long way to improving post quality. Doing this before the user posts instead of after is the key, and that's one of the great things about Discourse.

I think it should be the privilege of newbies to ask questions falsely. I remember finding even the links on the processing main page confusing. So it's great when they manage to find the forum, register and ask a question.

Remember: For some of the newbies it’s there first programming experience and there first time in a forum (any forum). So I think we want to be patient.

I don't disagree with that. I'm not saying we shouldn't help these users. I'm saying that the best way to help these users is to guide them through the process of solving their own problems. If a user posts "this is my homework what do I do" the answer is not "here is the code that you need to write". Instead, the answer is to ask them what they've tried so far. To help them break it down into smaller pieces. To teach them to take those pieces on one at a time instead of writing the whole thing at once.

"Give someone a fish and you feed them for a day; teach someone to fish and you feed them for a lifetime."

Chrisir

thanks for your good answer!

one point of mine was the possibility to upload entire projects

KevinWorkman

one point of mine was the possibility to upload entire projects

I don't think we should encourage this. We should encourage people to debug their code and to work in small isolated chunks. We should encourage them to narrow their problem down and to post a MCVE or an SSCCE instead of their full project.

This is not just helpful to us because it makes answering questions easier, but it's also helpful to the people asking the questions. These are good habits to get into regardless, and will only save headaches for everybody involved.

jeremydouglass

Well, MCVE or not (and I like MCVEs) many beginner sketches are processing (ahem) data from a companion file. As long as we are talking wish lists, it would be nice if it was a bit easier for them to provide linked resources when the provide their code -- often an image file or an audio file. I do a lot of demo sketches on the forum where I always point to the Processing3 logo file with a URL, but there isn't really an equivalent on the forum for an audio or video resource -- the closest is to use the filename of a video clip that can be copied from a built in demo sketch.

jeremydouglass

Another nice feature (not sure if it is available in Discourse) -- merge comment. Right now we have a lot of threads that get closed with "continued here" -- it might be better to just merge the continuation into the original.

quark

One of the poorly implemented features in this forum is split discussion where someone has started a new topic inside an existing discussions. When the moderator splits the discussion he/she becomes owner of the new discussion so get loads of notifications even if you are not interested in the topic.

KevinWorkman

Well, MCVE or not (and I like MCVEs) many beginner sketches are processing (ahem) data from a companion file.

I think that we should encourage posters to eliminate external files from the code they post. Don't include the whole file. Hard-code the section of file that causes a problem. We should have docs explaining this process.

As long as we are talking wish lists, it would be nice if it was a bit easier for them to provide linked resources when the provide their code -- often an image file or an audio file.

Although I don't think that we should encourage this, note that it is technically possible to allow attachments in Discourse: https://meta.discourse.org/t/new-attachments/8609

Another nice feature (not sure if it is available in Discourse) -- merge comment.

Yep: https://meta.discourse.org/t/merge-two-topics-into-one/9735

split discussion

Yep again: https://meta.discourse.org/t/can-an-admin-split-a-post-into-a-separate-topic/25373

REAS

I set up a "trial" site on Discourse. It's not fully configured with our custom categories, etc. but this gives a window into what it might be like.

http://processing.trydiscourse.com/

Chrisir

Thanks for your efforts!

It looks very promising!

Chrisir

BGADII

I'm not a fan of discourse at all, I'm new here but this forum is actually one of my favorites. It's easy on the eyes and comfortable. Everything seems to work and it's easy to find your way around. The only thing I don't like about the current version is I wish when you clicked notifications it would send you to the thread or a notifications page, not your profile or someone else's profile with no direct links to the thread in question.

KevinWorkman

I'm not a fan of discourse at all

Out of curiosity, what don't you like about it?

kfrajer

@KevinWorkman Discourse looks very close to your forum. Are they related?

Kf

KevinWorkman

@kfrajer Yep, I use Discourse for my forum!

BGADII

@KevinWorkman

Out of curiosity, what don't you like about it?

Overuse of empty space leads things to look unconstrained and hard to follow. Not enough navigational emphasis on sub-forums so everything tends to get lumped together. I understand the need to keep all recent discussions in the for-front for smaller forums, but it comes at a cost of organization. I think the way the current forum handles it is a good middle-ground, where all the forums/subforums are clearly listed on the left hand side always but the default homepage pulls from all forums. Discourse seems Reddit inspired.

A lot of my complaints with discourse could be fixed with a lot of modifications, but I've never seen a discourse forum that really nailed it from a UX standpoint, and anecdotally the culture on a lot of discourse boards I've seen becomes increasingly toxic as they grow because of the community moderation functions enable cliques and brigading through mentions.

Also, logging edit history publicly is a privacy concern that isn't necessary for forums which do not engage in group commerce.

jeremydouglass

Interesting points to consider.

Clear organization of sub-forums is something to take seriously, because posts about p5.js / processing.py etc. etc. are seldom if ever relevant to each other.

As a non-Reddit user I learned a new term: "brigading" -- https://www.reddithelp.com/en/categories/rules-reporting/account-and-community-restrictions/what-constitutes-vote-cheating-or

That said, I'm not too concerned about brigading in the forum -- the overwhelming majority of posts are by 1) a small cadre of volunteer answerers who all coexist with each other's quirks, or 2) individual students / developers / artists working on personal projects. Thus far in my experience there just aren't groups of people who socialize on here in a way that could form cliques or brigades.

KevinWorkman

Interesting. Thanks for sharing your opinion.

Not enough navigational emphasis on sub-forums so everything tends to get lumped together.

That's actually one of the things I prefer about Discourse. I think most forums (including this one) suffer from too many subforums: do we really need all of 37 them?

I personally don't agree with a lot of your other points, but I don't want to start a debate so I'll leave it there. Thanks again for sharing your opinion, even if it's different from mine!

GoToLoop

For moi, if this Vanilla forum can customize its markdown syntax for that of GitHub/Discourse's, w/ no need for trailing ending spaces for newline breaks, plus fence code blocks like this ```js, I see no need to move on to another forum system all over again! [-O<

quark

After playing with Discourse I found several other features well worth having -

1. Scrollable areas for large code blocks reducing the need to scroll the whole page.

2. Live preview.

3. Ability to email replies to discussions.

4. Last visit line

I am sure I will find others the more I play with it.

kfrajer

@jeremydouglass I said something like that right on my first post

What features do you want to get from stackoverflow? Do you want to bring the voting system? Please, don't... I don't find it useful in the long run.

Brigrading is not the only problem but one of them. People that help here have their own motives which suits the community. Waving in my face how many people I have helped in the last day, last week or last hour just doesn't cut it for me. I dislike those features. On the other hand, how does discourse manages multiple answers to a question? Wouldn't be great to select the best of the many so if anybody sees the post later on, he/she is able to tell which post was the actual answer (or which one to draw the attention to at first?)

Improving coding deployment would be a must feature that needs fixing int his forum (I am with @GoToLoop on this). Educating people will always need to happen no matter what...

Great discussion btw.

@REAS What would take to fix the current forum? I believe the code is available in github, right? Also, what is your vision for the time being related to moving to a new forum? What experiences did you have from previous roll-over operations?

Kf

KevinWorkman

On the other hand, how does discourse manages multiple answers to a question?

Out of the box, Discourse is not a QA site. It's a forum. So the "thinking in Discourse" argument is that people should read the whole discussion, not just look for a quick fix. And I think the best threads will indeed require discussion and back-and-forth instead of quick code dumps.

That being said, there is an official Discourse plugin that allows marking a post as the answer to the thread: https://meta.discourse.org/t/discourse-solved-accepted-answer-plugin/30155

noel

I would like to see a complete search for tags.

phoebus

"PSpace is the simplest possible virtual co-working space for the Processing community. It consists of a single web app/page combining: -minimal WebGL multi-user 3D scene -voice chat interface -text chat interface "

REAS

I first want to thank everyone for thinking this through with us and for trying out the Discourse test site. We are planning to make the switch to Discourse in mid April after the Google Summer of Code application period ends and to give us a few weeks to prepare.

We've weighed the positives and negatives about the move based on the discussion and comments and we feel it's the right thing to do.

KevinWorkman

Awesome! I think this is definitely the right thing to do.

Any thoughts on coming up with policy documentation or customizing the messaging that Discourse automatically shows to new users? I'd be happy to help put some of that together.

REAS

That's a part of the "a few weeks to prepare" from the message above. :)

Yes, please, if you can look into that, it would be great. The default statement from Discourse felt good from my skim through it. The other document to consult is the p5.js Community Statement.

jeremydouglass

How will this forum be archived? It would be great if we could switch off relative time-date stamps before making it static. The announcement for Forum two has been "16 days ago" for the last five years.

https://forum.processing.org/one/topic/new-forum-wednesday-night-gmt-4-this-forum-will-be-locked.html

hamoid

@jeremydouglass Is the forum one in html format? It would not be so hard to fix that with some regexp and command line magic :) The actual date is contained in the link:

<em class="ndboldem" title="01-Oct-2013 03:14 PM"> 16 days ago </em>

If I can access the data I volunteer to do it.

hamoid

Even simpler: inject this JavaScript code into one of the JS files loaded in the old forum to replace all relative dates with absolute dates:

$(function() {
  $('.ndboldem').each(function(i) { 
    var dte = $(this).attr('title'); 
    if(/^\d{2}-\w{3}-\d{4} \d{2}\:\d{2} \w{2}$/.test(dte)) {
      $(this).html(dte); 
    }
  });
});