Royal penguins arguing

As previously discussed, I am currently in the process of remaking Bicker with modern technologies and architecture, hopefully along with lessons learned over the last decade about what makes a good community system. In the interest of transparency, I’m using the blog as a kind of developer journal while I work, posting occasional updates as to where things stand.

If you want to read the other posts in the series, you can get a full list of posts under the “bicker” tag.

Paragraph Blocks

As mentioned last time, one of the nagging bugs is that Markdown is, in many cases, a “modal” language. That is, there are situations where a certain kind of formatting is true until the mode termination is issued.

The big example is a code block, which looks something like this:

    ```
    Some code
    ```

If we separate that into individual lines, the code is no longer in the relevant mode, and so each line displays as normal text, as if there was no formatting intent.

Tables are similar, where each line of a table needs to be treated as a whole, so those lines also need to be packaged together.

…need to track those same modes while separating the text…

This complicates the message creation code, because we now need to track those same modes while separating the text into paragraphs. It’s not exactly pretty—with a complicated conditional run against every line simulating a simple state machine—but can always be refactored into a more reasonable parser some other time.

Two syntax details that kept tripping me up, though:

  • No matter how many times I type it, '\n' is not a new-line character. Escape characters like that only work in double-quoted strings, such as "\n".
  • Since I tripped up last time over the difference between = and ==, along with my surprise to remember that Ruby still has a lot of C-isms along those lines, it shouldn’t be all that surprising that I assumed logical-and and logical-or would be C’s && and ||. Nope. This time, Ruby smartly opts for the improved readability of and and or.

I point this out, in part, to show the less-experienced programmers that we all foul up. But I mostly just hope that I can somehow shame myself into not making the same mistake every single time it comes up…

Avatars

After that, in order to prepare for replies to messages, it helps to be able to identify who said what. Besides just showing the paragraphs with a name (and probably a time posted) in a separate column, we will probably also want user avatars, so we might as well get that out of the way now.

…Libravatar works well…

As it turns out, Libravatar works well—especially after their recent overhaul—and the libravatar gem, while a little bit old, does almost all of the heavy lifting. Some of the available kinds of avatars look nice, too.

Avatar

To make it work, we need a helper function, so that it can be used anywhere in the system. Such things, I suppose unsurprisingly, live in application_helper.rb.

require 'libravatar'
#...
def avatar sz = 50, user = current_user
  libra = Libravatar.new({
    :email => user.email,
    :size => sz,
    :https => true,
    :default => 'robohash'
  })
  libra.to_s
end

And once we have that, the views can now add a little identification box to every paragraph.

<p class="msg-par-avatar">
  <% user = User.find_by_id(para.user_id) %>
  <%= link_to image_tag(avatar 100, user), '/' %>
  <br>
  <span class="id-time"><%= user.login %></span>
  <br>
  <span class="id-time"><%= para.created_at %></span>
</p>

I’ll figure out where the avatar should link some other time and deal with formatting the timestamp to something sensible. But until then, we now get a unique avatar image for each user and motivated users can opt to upload a specific image to Libravatar, if they so choose. So, that seems sufficient.

…no idea if the federation is up to the task…

Note that Libravatar is both open source and federated. At least in theory, this should mean that Bicker users aren’t exposed to any unnecessary privacy risk, because they can run their own server with their avatar(s). Unfortunately, I have no idea if the federation is up to the task or if the gem (which, at the time I write this, has gone for nearly four years without a release) has implemented any of the federating features. That will require more investigation in the future.

If Libravatar turns out to not be up to snuff, it looks like Gravatar has almost the same API and support, though I’d rather not unnecessarily lean on a company. Automattic certainly seems to be on the right side of history, so using one of their products wouldn’t be the worst choice, by any means, but they do still have the same vulnerabilities of any company taking venture capital and run a lot of centralized projects well-positioned for non-intrusive surveillance, Gravatar (along with Akismet) near the top of that list.

Deleting Messages

After getting the formatting and avatars in place, I realized that I never updated the message controller to handle the paragraphs beyond displaying them. And since the paragraphs are “threaded,” that requires some work to make sure we don’t violate foreign key constraints.

Specifically, the set of paragraphs for the message need to be assembled from the paragraph whose next_id is nil, working backwards to the paragraph that has no paragraph pointing to it. In other words, we need to find a linked list in a pool of nodes.

linked list

Once we have that list, we can iterate from beginning to end, deleting each paragraph in an order that doesn’t violate any key constraints. When that has finished, we can delete the message itself. And not a moment too soon, either, considering that testing the formatting generated dozens of bogus messages in my database while I figured out the right sequence…

I’ll need to take another pass at this, once we have replies—replies are paragraphs that point to the paragraph they’re replying to as a “parent”—but that’s a straightforward recursive process.

Touchup

I made some smaller user interface changes, such as changing the index layouts and adding a list of relevant messages when displaying a category.

Messages, likewise, can only be in categories of messages and now must include a subject and text to be valid. Somehow, the latter has slipped through the cracks.

Don’t Over-React

…development certainly hasn’t gotten harder…

Now that we’re about ready to put some real work into the user-interface, it’s time (maybe a bit past time) to think a little bit about using a front-end application framework instead of hand-coding Ruby’s templates. Hand-coding will work, of course. I know, because I did exactly that, about ten years ago, and development certainly hasn’t gotten harder in the intervening time.

Because I’ve had pretty good luck with it on other projects such as SlackBackup, I’m going with React. There’s a learning curve for new developers, of course, but the components make for much easier maintenance.

There are many articles about adding React to Rails projects. But the clearest, one of the few I see that strips the process down to just the important parts instead of putting the focus on the demo project being built, is from PluralSight, specifically the Setting up the controllers and Adding React to Rails sections.

See below, though: It was much more effective to read the project documentation than any of the articles I found.

To condense it further, we add the responders and react-rails gems to the project Gemfile, then install.

bundle install
rails webpacker:install:react
rails generate react:install

Create the base_controller.rb and site_controller.rb. Use their items_controller.rb as a template for the real controllers. Add the new routes.

The application layout (app/views/layouts/application.html.erb) needs some extra JavaScript tags added.

<%= javascript_pack_tag 'application' %>

And then the new views will replace or supplement their old contents with…

<%= react_component 'ComponentName', {
  property1: value1,
  property2: value2,
} %>

And from there, React will do its thing.

If you’re not already familiar with React, it’s probably easiest to think of React components as functions: You hand them data as input (from the view in a syntax resembling HTML properties) and they return a piece of the interface, where any of those pieces may include other React components.

If you are already familiar with React, that paragraph probably just offended you by over-simplifying. Sorry!

We then create the components using a fairly convenient generator:

rails generate react:component TheComponent name:string

When I say convenient, what I specifically mean is that, when you provide the generator with a “schema” (just name:string, in this example), the resulting React component accepts those fields as properties and type-checks them.

…that’s where to go for information…

Unfortunately, the article cited above doesn’t mention this, and instead provides a bunch of bad (maybe outdated?) advice about creating the components manually. Instead, the generator is described in the project’s repository README, which also goes into a lot more depth on issues such as using TypeScript in components. I won’t be doing that, here, but that’s where to go for information on doing so. It’s a real shame that the README isn’t higher in the search results, all things considered.

New Paragraphs

Regardless of my caveats and second-guessing, the above rundown should work…at least, as of today. So, at the very least, we need to replace the existing paragraph HTML with a React component. For example…

rails generate react:component Paragraph avatar:string content:string ts:string when:string who:string

I could probably do a lot better on the types, there, rather than making everything a string. But we already have Rails doing a lot of the formatting, so it doesn’t yet make much sense to worry about where it should “really” land in the end. After tracking down the don’t-do-this way of inserting raw HTML into a React component, we get a component like this.

class Paragraph extends React.Component {
  render () {
    return (
      <React.Fragment>
        <div class="msg-paragraph">
          <p
            class="msg-par-text"
            dangerouslySetInnerHTML={ this.props.content }
          >
          </p>
          <p class="msg-par-avatar">
            <a href='/'><img src={ this.props.avatar } /></a>
            <br />
            <span class="id-time">{ this.props.who }</span>
            <br />
            <span
              class="id-time"
              title={ this.props.ts }
            >
              { this.props.when } ago
            </span>
          </p>
        </div>
      </React.Fragment>
    );
  }
}

And we insert it with…

<%= react_component 'Paragraph', {
  avatar: avatar(100, user),
  content: {
    __html: para.content
  },
  ts: para.created_at.to_formatted_s(:db),
  when: time_ago_in_words(para.created_at),
  who: user.login
} %>

Yep. We have dangerouslySetInnerHTML and __html to contend with. We’ll probably want to work around that, at some point. On the other hand, I like the new time formatting and it wasn’t at all hard to extract the old paragraph markup and convert it to React.

From here, we now have a foothold in the application with React and are ready to finally work on our replies.

Punctuation Decoration Situation

And now that we finally have a component-based user interface, I can start the process of allowing replies. As I mentioned in Bicker’s introductory post, Bicker’s central moderation feature is…

…only allowing responses to be inserted after a punctuation mark in someone else’s post.

The way we do that is by making every punctuation mark a reply to this button.

These days, there are plenty of ways to accomplish this. For example, we could write JavaScript to manipulate the DOM to insert what we need dynamically. However, far less intensive seems to be to leave it to the React paragraph component to replace punctuation with the buttons.

…split the paragraph into its components…

Or, rather, we’ll split the paragraph into its components in Ruby and then pass that to React to be fit into the proper kinds of components. While I’ve never seen anybody else use the term, I find it easy to talk about using front-end frameworks on top of web frameworks as “Iterated MVC, so our Rails view is now—in this sense—React’s controller.

To chop up a paragraph, then, we’re going to need to know how to hack it up, which means deciding what constitutes a punctuation mark. My preliminary list is going to be:

  • .
  • ,
  • !
  • ?
  • ;
  • :
  • (
  • )
  • &
  • &ndash; (–)
  • &mdash; (—)
  • &hellip; (…)

There are probably others that should be added, but the list can always be expanded later and this should work for most conversations in English and similar languages, which will almost certainly be most of my initial testing.

…ignore the ampersand and semi-colon for now.

The easiest way to get Ruby to separate text based on a set of delimiters and retain the delimiters is probably regular expressions. Because we’ll be replacing the punctuation and because we need to deal with those HTML entities, it makes sense to ignore the ampersand (&) and semi-colon (;) for now. It’s not like you’ll see a lot of people actually using the former in place of the word “and,” at this point, and a lack of semi-colons is going to hurt me more than anybody else. So, we end up with…

if !paragraph.content.starts_with?('<code>') and
   !paragraph.content.starts_with?(/<h[1-6]/)
  paragraph.content
    .gsub!(
      /(\.|,|!|\?|:|\(|\)|&ndash;|&mdash;|&hellip;)/,
      "\n".concat('\1').concat("\n")
    )
    .split("\n")
else
  paragraph.content = [ paragraph.content ]
end

That comes pretty close, but it tears apart things like URLs, meaning that we need to either find a way to dodge around HTML tags or plow through making the replacements manually. Weirdly, it also looks like this might be trashing our tables, somehow.

We are also going to want to exclude certain kinds of paragraphs from processing, such as headers, code, and tables. If we insert a reply into any of them, we’ll inevitably destroy the formatting.

For our trouble, we end up with paragraphs that look something like…

Our paragraph

With the caveats above, we now have our not-entirely-attractive buttons, or at least reasonable placeholders that will become buttons in a future version.

Next

That’s about all the time I had for Bicker, this week.

Coming up, I need to work out the issues with HTML entities and tags, before I do anything else.

Then, the next big change will obviously be adding the form to submit replies and—time permitting—managing the replies from there. It’s also getting close to time to dealing with roles; Bicker already has Rolify installed, so it’s a matter of making use of it.


Credits: The header image is Royal penguins arguing by Brocken Inaglory and is made available under the terms of the Creative Commons Attribution-Share Alike 3.0 Unported License. The linked list by Lasindi has been released into the public domain.