If builds for both platforms need to be given to the customer on the same day, they are released at the same time and there is only one QA engineer on the project, there may not be enough time for verification in native development: the testing cycle must be performed on both platforms separately. When testing cross-platform applications, we save time: regression testing of both platforms takes place within a single cycle.

We tried to roughly evaluate the testing of two similar projects - one on native Android and iOS, the second on Flutter - and compared them feature-by-feature.

Analysis and testing were carried out on one iOS device and on one Android platform device. As you can see in practice, Flutter really gives a gain in time, although not twice. This is understandable: it is impossible to completely remove testing on one of the two platforms. Like it or not, they have different specifics and user focus.

The logic of new features on both platforms is also the same, since one code is written: testing complex processes inside the application comes down to testing them on one platform and confirming them on another. We conduct a full cycle of activities on one platform: we do exploratory testing, feature runs, smoke / sanity / full, analyze feedback. After that, it remains only to confirm the quality by exploratory testing on another platform. This approach saves the time of testing the logic by about 1.3 times.

Example

Suppose that analysts, while testing events, noticed a situation: according to the TOR, the event should be sent to the appropriate analytics system, but it is not tracked. Such a gap in the logic clearly indicates that the event will not be sent on both platforms.

Having fixed the bugs and made sure in various scenarios that the collection of analytics in the application works correctly on one of the platforms (for example, iOS), we can assume with greater confidence that we will not encounter a bug on another platform (Android) associated with the track of an extra event or the absence of the necessary event in the analytics system.

Reusable Components

Removing reusable components is now quite simple. Both types of tests use the same class - WidgetTester. Therefore, there are no problems in the quality assurance with the operation of selectors or generic functions. We take them out separately, for example, and do not know grief.

In summary: we will reuse selectors (Finder class), functions, and gestures between Widget and E2E tests.

class AuthScreen {

 // Screen for entering a pin code during authorization

 static Finder pinScreen = find.byKey(AuthTestKeys.pinScreen);

 // Set pin screen

 static Finder pinCreateScreen = find.byKey(AuthTestKeys.pinCreateScreen);

 // Pin-keyboard element when entering a pin-code during authorization

 static Finder pinNumberLogin(String number) =>

     find.descendant(of: pinScreen, matching: find.byKey(AuthTestKeys.pinKeyboardBtn(number)));

 // Login field on the login screen

 static Finder loginField = find.byKey(AuthTestKeys.loginField);

< . . . >

}

An important clarification: let's say there are several steps that are similar to each other. One step entirely contains the other: "I tap the button" and "I tap the button twice." This causes a problem: the implementation of the step that was found first is determined. In this case, it is important to use regular expressions (as in other cross-platform tools). RegExp with $ at the end - RegExp('I'm pressing the $ button') - helps to avoid conflicts, and clearly shows where the end of the step is. There is no longer a situation where the matcher chose the wrong step definition.

Widget script for implementation

testWidgets('Max field length is 11 characters', (WidgetTester tester) async {

// given

      await tester.pumpWidget(GeneralWidgetInit.defaultWrapper(

        const AuthLoginScreen(),

      ));

      await tester.pumpAndSettle();

      await tester.doEnterText(AuthScreen.phoneFld, randomNumber(12));

// then

      final field = tester.widget<TextField>(AuthScreen.phoneFld);

      expect(field.controller.text.length, 11);

    });

< . . . >

 }); // group('Authorization by phone')

}

< . . . >

Previously, in order to refer to an element, it was impossible to determine it by a locator during quality assurance process, so that it was suitable for both the Widget test and the E2E test. Now it is available automatically.

Conveniently, you can take an integer property of the Widget—like the length—and compare it to the expected result. When checking field restrictions, enter text greater than 11 characters. At the output, we check that only 11 was entered.