[POC] Integrate the E2E test utils with `mattermost-plugin-todo` project

[panoramix360]

hey!

I'll post my discoveries in the mattermost-plugin-e2e-test-utils integration with the mattermost-plugin-todo project here.

The idea behind this is:

• Create utility code here in this project that can be used throughout other mattermost plugins that want to test the projects using Playwright
• The utility code will have things like starting a container with mattermost server that can be used in e2e tests, log in automatically, and stuff like that

Some of the initial code is being based on this PR mattermost/mattermost-plugin-msteams#459

The branch I created to test these things is: feature/playwright-setup

Feel free to take a look at it right now.

I'll write a next comment/post with the initial discoveries.

[panoramix360]

Mattermost Plugin E2E Test Utils Configuration

In the branch created I added a playwright configuration using the normal setup in the Playwright documentation.

After that, I added the utility code that was in the based PR reference in the above post that includes the testcontainers library and other libraries that support it like the pg one.

This code creates Docker containers behind the scenes to start up a Mattermost server container to test things. (I'll need to confirm that yet)

And then I exported these utilities on the index.ts file to be used by others when referencing this package in npm.

Mattermost Plugin Todo

After doing these initial configurations described above, I was able to reference the e2e test utils project inside the mattermost-plugin-todo and use the utility code there like the MattermostContainer and the RunContainer classes.

I did an initial playwright configuration just like the above on a playwright folder inside the mattermost-plugin-todo project.

After that, I created a demo-test.spec.ts on this folder that has some tests to use the mattermost-plugin-todo.

But when I ran the test, I had a problem because the RunContainer class was referencing a dist folder that I didn't know what that was, this is the code I'm referring to:

const RunContainer = async (): Promise<MattermostContainer> => {
  let filename = "";
  fs.readdirSync("../dist/").forEach(file => {
      if (file.endsWith(".tar.gz")) {
          filename = "../dist/"+file
      }
  })
  if (filename === "") {
      throw("No tar.gz file found in dist folder")
  }

From what I understood, this seemed to be a .tar.gz that comes from the com.mattermost.msteams-sync project, so I'll probably need to understand and change that so this can work as expected.

Probably we will need to make this agnostic of the project and this structure will need to change.

That's what I'm going to work on right now.

My initial idea with this is to try to create the simplest version of the PR and run the tests just to see if this works, so now I'll probably need to work on configuring and changing more things to make a simple TODO plugin test pass.

[mickmister]

But when I ran the test, I had a problem because the RunContainer class was referencing a dist folder that I didn't know what that was, this is the code I'm referring to:

The dist folder is generated in the root directory of the plugin repository (any plugin repository), when the plugin is compiled with make dist or make deploy etc. A bundle in the form of (plugin id).tar.gz is created, that can then be uploaded to the Mattermost server to run there.

So in order for fs.readdirSync("../dist/") to succeed, we'll need to have a established convention of the location of the folder we run the tests from. At the moment, it seems RunContainer assumes we are "one folder" into the repo, as .. would lead us to the root directory, and ../dist would lead us to dist.

My initial idea with this is to try to create the simplest version of the PR and run the tests just to see if this works, so now I'll probably need to work on configuring and changing more things to make a simple TODO plugin test pass.

Awesome 🎉 Feel free to braindump here if anything comes up that you want to discuss 👍

Excellent work Lucas!!

[panoramix360]

Nice, thanks for your thought.

Do you have any idea how we should approach this? since the .tar.gz file will be different from plugin to plugin

Maybe we would need to pass a path for the readdirSync to read inside the plugin folder.

I'll try to come up with something easy.

[mickmister]

Do you have any idea how we should approach this? since the .tar.gz file will be different from plugin to plugin

@panoramix360 I'm not sure I understand. The code you linked above makes no assumptions about the plugin id right? I think it should work with every plugin, as long as the e2e test directory structure is consistent among the projects.

[panoramix360]

Not on this part that I inserted here, but in another place it has a reference to the mteams plugin, but that's not the issue.

The issue here is:

• This code above assumes the dist folder needs to be inside the mattermost-plugin-e2e-test-utils but that's not actually what we want to do, we have to use the dist/ folder inside the project that it's using this repo.

But I'll think about something, so we can have a path passed by an argument maybe that uses an absolute path to read the file.

What do you think?

[mickmister]

• This code above assumes the dist folder needs to be inside the mattermost-plugin-e2e-test-utils but that's not actually what we want to do, we have to use the dist/ folder inside the project that it's using this repo.

@panoramix360 I'm not sure if it works that way with a relative path. When we run fs.readdirSync("../dist/"), I believe that is relative to the CWD of where we are in the terminal when we run the tests. If we involved __dirname in this calculation, it would be relative to this file, but since we just do ../dist, it should simply be the parent directory of the CWD.

So if we run npm test in mattermost-plugin-todo/e2e/playwright, we would want to do ../../dist

[jespino]

@panoramix360 If you want we can have a meeting to give you my POV about it, and even do some pairing to move this forward.

[panoramix360]

hey @jespino and @mickmister

first, it seems that the relative path of readdirSync is using the directory inside the e2e-utils and not on the plugin-todo, but I'll test more today.

@jespino, I'd be glad to have a meeting about it :D can we schedule on Mattermost Community? maybe @mickmister will want to participate as well

[jespino]

@panoramix360 sure, let's do it.

[panoramix360]

@mickmister it worked!

I managed to pass the path to the dist folder and it used the dist folder inside the todo-plugin so that's good :D

[panoramix360]

hey!

I configured the tests in a branch inside the mattermost-plugin-todo with the playwright configurations and it's referencing the mattermost-plugin-e2e-test-utils locally.

And this is the initial test:

import { test, expect } from "@playwright/test";

import {
  RunContainer,
  MattermostContainer,
  login,
  logout,
} from "mattermost-plugin-e2e-test-utils";

let mattermost: MattermostContainer;

test.beforeAll(async () => {
  mattermost = await RunContainer({
    packageName: "com.mattermost.plugin-todo",
    distPath: "../dist",
  });
});

test.afterAll(async () => {
  await mattermost.stop();
});

test.describe("link slash command", () => {
  test("try to link a channel as regular user", async ({ page }) => {
    const url = mattermost.url();
    await login(page, url, "regularuser", "regularuser");
    await expect(page.getByLabel("town square public channel")).toBeVisible();
    await logout(page);
  });
});

This test is passing 🥳

@mickmister what do we go from here? maybe creating more tests so we can have some parts of the plugin-todo tested just as a reference?