MINOR: Simplify reassignment logic

This patch has a few simplifications to the reassignment logic:

Simplify the ReassignmentContext bookkeeping so that we do not need to track the previous reassignment.
Remove need to track the source of the reassignment (i.e. zookeeper or API) in the context.
Ensure reassignment context in memory is consistent with the persisted reassignment state.
Fix some inconsistency in how we handle concurrent zk and api reassignments. Specifically, we have added logic to explicitly cancel individual partition reassignments in zk when an overriding api call is received.

Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

cc @stanislavkozlovski Can you take a close look at the updates? I think the main changes are the following:

I've removed the intermediate state when overriding a previous reassignment. Instead, the logic now jumps directly from the old reassignment to the new one and stops any replicas left in the dust.
Upon reassignment completion for a partition, the logic now always checks for a zk partition reassignment to be removed. This prevents any inconsistencies as a result of concurrent reassignments.
Api reassignments explicitly remove reassignment state from zk prior to beginning the new reassignment. I thought it was a little dangerous to allow both the zk reassignment and the api reassignment to exist at the same time.

@hachikuji Would you consider these a blocker for 2.4? cc @omkreddy

@viktorsomogyi Thanks for reviewing. I'll file a jira. Initially this was intended to be a minor fix, but it grew as I started digging into the logic.
@ijuma I wouldn't call it a blocker. I think the patch makes the logic more robust and hopefully easier to follower, but there is not a specific known problem that it fixes.

Jenkins hit a couple OOMs before running. I built successfully against 2.11 and 2.12 locally. I will go ahead and merge.


Suggestion: How to configure Webpack Monitor with Mix + Laravel

I am not very strong when it comes to bolting build tools together and I am probably not the only one :)
Would it be possible to include a basic how-to? Maybe something that would watch the default Laravel example here.
Thanks!

Well, that was easy :)
const WebpackMonitor = require('webpack-monitor');

###

mix.webpackConfig({
    plugins: [
        new WebpackMonitor({
            capture: true,
            launch: true,
        }),
    ],
})


Migration Exception "LUB/GLB with unexpected types: DynamicTypeImpl/InterfaceTypeImpl"

Description
I am using RxDart (v0.26.0) ValueStreamExtensions.value getter extension. Here is the extension definition in the library.
// rxdart-0.26.0/lib/src/streams/value_stream.dart
extension ValueStreamExtensions<T> on ValueStream<T> {
  /// Returns last emitted value, or null if there has been no emission yet.
  T? get value => valueWrapper?.value;
}

Here is how I use it in one of my methods.
import 'package:rxdart/rxdart.dart';

class DefaultFileUploader {
  final BehaviorSubject<List<UploadHandler>> handlersStream =
      BehaviorSubject<List<UploadHandler>>.seeded([]);

  void forceUpdateHandlerStream() {
    final List<UploadHandler> currentHandlers = handlersStream.value ?? []; /// error line
    handlersStream.add(currentHandlers);
  }
}

Error Occurred
When I run dart migrate the following error occurs. My personal guess is the migrate tool is not configured to handle extension method getter with a ?? null check operator.

Aborting migration due to an exception.  This most likely is due to a
bug in the migration tool.  Please consider filing a bug report at:
https://github.com/dart-lang/sdk/issues/new
Please include the SDK version (2.12.2) in your bug report.
To attempt to perform migration anyway, you may re-run with
--ignore-exceptions.
Exception details:
UnimplementedError: LUB/GLB with unexpected types: DynamicTypeImpl/InterfaceTypeImpl in "handlersStream.value ?? []" on line 53 of "/home/nipuna/panda/flutter_projects/project-manager/lib/managers/upload_manager/definitions/default_file_uploader.dart" at offset 1777 in /home/nipuna/panda/flutter_projects/project-manager/lib/managers/upload_manager/definitions/default_file_uploader.dart (handlersStream.value ?? [])
#0      EdgeBuilder._unimplemented (package:nnbd_migration/src/edge_builder.dart:3253:5)
#1      EdgeBuilder._decorateUpperOrLowerBound (package:nnbd_migration/src/edge_builder.dart:2129:11)
#2      EdgeBuilder._decorateUpperOrLowerBound (package:nnbd_migration/src/edge_builder.dart:2058:14)
#3      EdgeBuilder.visitBinaryExpression (package:nnbd_migration/src/edge_builder.dart:495:26)
#4      BinaryExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:967:49)
#5      EdgeBuilder._dispatch (package:nnbd_migration/src/edge_builder.dart:2207:24)
#6      EdgeBuilder._handleAssignment (package:nnbd_migration/src/edge_builder.dart:2329:20)
#7      EdgeBuilder.visitVariableDeclarationList (package:nnbd_migration/src/edge_builder.dart:1924:11)
#8      VariableDeclarationListImpl.accept (package:analyzer/src/dart/ast/ast.dart:10655:15)
#9      EdgeBuilder._dispatch (package:nnbd_migration/src/edge_builder.dart:2207:24)
#10     EdgeBuilder.visitNode (package:nnbd_migration/src/edge_builder.dart:1340:9)
#11     GeneralizingAstVisitor.visitStatement (package:analyzer/dart/ast/visitor.dart:517:39)
#12     GeneralizingAstVisitor.visitVariableDeclarationStatement (package:analyzer/dart/ast/visitor.dart:589:7)
#13     VariableDeclarationStatementImpl.accept (package:analyzer/src/dart/ast/ast.dart:10705:15)
#14     EdgeBuilder._dispatch (package:nnbd_migration/src/edge_builder.dart:2207:24)
#15     EdgeBuilder.visitNode (package:nnbd_migration/src/edge_builder.dart:1340:9)
#16     GeneralizingAstVisitor.visitStatement (package:analyzer/dart/ast/visitor.dart:517:39)
#17     GeneralizingAstVisitor.visitBlock (package:analyzer/dart/ast/visitor.dart:165:31)
#18     BlockImpl.accept (package:analyzer/src/dart/ast/ast.dart:1083:49)
#19     EdgeBuilder._dispatch (package:nnbd_migration/src/edge_builder.dart:2207:24)
#20     EdgeBuilder.visitNode (package:nnbd_migration/src/edge_builder.dart:1340:9)
#21     GeneralizingAstVisitor.visitFunctionBody (package:analyzer/dart/ast/visitor.dart:324:45)
#22     GeneralizingAstVisitor.visitBlockFunctionBody (package:analyzer/dart/ast/visitor.dart:168:55)
#23     BlockFunctionBodyImpl.accept (package:analyzer/src/dart/ast/ast.dart:1038:49)
#24     EdgeBuilder._dispatch (package:nnbd_migration/src/edge_builder.dart:2207:24)
#25     EdgeBuilder._handleExecutableDeclaration (package:nnbd_migration/src/edge_builder.dart:2488:7)
#26     EdgeBuilder.visitMethodDeclaration (package:nnbd_migration/src/edge_builder.dart:1243:5)
#27     MethodDeclarationImpl.accept (package:analyzer/src/dart/ast/ast.dart:7033:49)
#28     EdgeBuilder._dispatch (package:nnbd_migration/src/edge_builder.dart:2207:24)
#29     EdgeBuilder._dispatchList (package:nnbd_migration/src/edge_builder.dart:2229:7)
#30     EdgeBuilder.visitClassOrMixinOrExtensionDeclaration (package:nnbd_migration/src/edge_builder.dart:607:7)
#31     EdgeBuilder.visitClassDeclaration (package:nnbd_migration/src/edge_builder.dart:574:5)
#32     ClassDeclarationImpl.accept (package:analyzer/src/dart/ast/ast.dart:1536:49)
#33     EdgeBuilder._dispatch (package:nnbd_migration/src/edge_builder.dart:2207:24)
#34     EdgeBuilder.visitNode (package:nnbd_migration/src/edge_builder.dart:1340:9)
#35     GeneralizingAstVisitor.visitCompilationUnit (package:analyzer/dart/ast/visitor.dart:202:51)
#36     CompletenessTracker.visitCompilationUnit. (package:nnbd_migration/src/utilities/completeness_tracker.dart:52:24)
#37     PermissiveModeVisitor.reportExceptionsIfPermissive (package:nnbd_migration/src/utilities/permissive_mode.dart:26:24)
#38     CompletenessTracker.visitCompilationUnit (package:nnbd_migration/src/utilities/completeness_tracker.dart:43:5)
#39     CompilationUnitImpl.accept (package:analyzer/src/dart/ast/ast.dart:2134:49)
#40     NullabilityMigrationImpl.processInput (package:nnbd_migration/src/nullability_migration_impl.dart:244:12)
#41     NonNullableFix.processUnit (package:nnbd_migration/src/front_end/non_nullable_fix.dart:164:15)
#42     _FixCodeProcessor.runLaterPhases. (package:nnbd_migration/migration_cli.dart:1154:19)
#43     _FixCodeProcessor.runLaterPhases. (package:nnbd_migration/migration_cli.dart:1152:28)
#44     _FixCodeProcessor.processResources (package:nnbd_migration/migration_cli.dart:1081:30)
#45     _FixCodeProcessor.runLaterPhases (package:nnbd_migration/migration_cli.dart:1152:5)
#46     MigrationCliRunner.run (package:nnbd_migration/migration_cli.dart:749:24)
#47     MigrateCommand.run (package:nnbd_migration/migration_cli.dart:185:7)
#48     CommandRunner.runCommand (package:args/command_runner.dart:196:13)
#49     DartdevRunner.runCommand (package:dartdev/dartdev.dart:199:18)
#50     runDartdev (package:dartdev/dartdev.dart:53:16)
#51     main (file:///tmp/tmpTqI70l/dart-2.12.2/dart/pkg/dartdev/bin/dartdev.dart:11:3)

System Information
Here's my Flutter doctor output:

[✓] Flutter (Channel unknown, 2.0.3, on Linux, locale en_US.UTF-8)
• Flutter version 2.0.3 at /home/nipuna/fvm/versions/2.0.3
• Framework revision 4d7946a68d (2 weeks ago), 2021-03-18 17:24:33 -0700
• Engine revision 3459eb2436
• Dart version 2.12.2
[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
• Android SDK at /home/nipuna/Android/Sdk
• Platform android-30, build-tools 30.0.2
• ANDROID_HOME = /home/nipuna/Android/Sdk
• Java binary at: /home/nipuna/.local/share/JetBrains/Toolbox/apps/AndroidStudio/ch-0/193.6626763/jre/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
• All Android licenses accepted.
[✓] Chrome - develop for the web
• CHROME_EXECUTABLE = /usr/bin/google-chrome-stable
[✓] Android Studio (version 4.0)
• Android Studio at /home/nipuna/.local/share/JetBrains/Toolbox/apps/AndroidStudio/ch-0/193.6626763
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
[✓] IntelliJ IDEA Ultimate Edition (version 2020.3)
• IntelliJ at /home/nipuna/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/203.6682.168
• Flutter plugin version 55.0.3
• Dart plugin version 203.7759
[✓] IntelliJ IDEA Ultimate Edition (version 2020.2)
• IntelliJ at /home/nipuna/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/202.8194.7
• Flutter plugin version 51.0.3
• Dart plugin version 202.7319.5
[✓] Connected device (2 available)
• Android SDK built for x86 (mobile) • emulator-5554 • android-x86    • Android 6.0 (API 23) (emulator)
• Chrome (web)                       • chrome        • web-javascript • Google Chrome 88.0.4324.150
• No issues found!

I will post more information if required, please ask :)

@nipunasudha thanks for the detailed information!  I believe this is enough information that I will be able to reproduce and fix the bug.
Were you able make progress by re-running migration with --ignore-exceptions?

Migration tool runs successfully with no errors when I use --ignore-exceptions. Migration is applied successfully, with no changes to the line with the ?? operator.


Portal License not applying with "The portal site has not been upgraded. Please upgrade the site and try again."

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request

Module Version

3.3.0

Affected Resource(s)
ArcGIS_Portal
Configuration Files

{
    "AllNodes": [{
            "NodeName": "VMCLIENTDS1",
            "DataStoreTypes": [
                "Relational"
            ],
            "Role": [
                "DataStore"
            ]
        }, {
            "NodeName": "VMCLIENTWEB1",
            "Role": [
                "ServerWebAdaptor",
                "PortalWebAdaptor"
            ]
        }, {
            "NodeName": "VMCLIENTMAPS",
            "Role": [
                "Portal"
            ]
        }, {
            "NodeName": "VMCLIENTAGS1",
            "Role": [
                "Server"
            ]
        }
    ],
    "ConfigData": {
        "Version": "10.9.1",
        "ServerContext": "agsmapservice",
        "PortalContext": "arcgis",
        "ServerRole": "GeneralPurposeServer",
        "Credentials": {
            "ServiceAccount": {
                "Password": "secret",
                "UserName": "DOMAIN\\installer",
                "IsDomainAccount": true,
                "IsMSAAccount": false
            }
        },
        "Server": {
            "LicenseFilePath": "E:\\administration\\licenses\\10.9.1\\ArcGISGISServerStandard_ArcGISServer_1136407.prvc",
            "Installer": {
                "Path": "E:\\apps\\content\\Installs\\ArcGIS 10.9.1\\ArcGISServer\\Setup.exe",
                "InstallDir": "E:\\apps\\esri\\Server",
                "InstallDirPython": "E:\\apps\\python\\python-27\\",
                "EnableArcMapRuntime": "True",
                "EnableDotnetSupport": "False"
            },
            "ServerDirectoriesRootLocation": "E:\\agstore\\arcgisserver\\directories",
            "ConfigStoreLocation": "E:\\agstore\\arcgisserver\\config-store",
            "PrimarySiteAdmin": {
                "UserName": "admin",
                "Password": "secret"
            }
        },
        "Portal": {
            "LicenseFilePath": "E:\\administration\\licenses\\10.9.1\\ArcGIS_Enterprise_Portal_1091_374355_20220203.json",
            "PortalLicenseUserTypeId": "creatorUT",
            "Installer": {
                "Path": "E:\\apps\\content\\Installs\\ArcGIS 10.9.1\\PortalForArcGIS\\Setup.exe",
                "InstallDir": "E:\\apps\\esri\\Portal",
                "ContentDir": "E:\\agstore\\arcgisportal"
            },
            "ContentDirectoryLocation": "E:\\agstore\\arcgisportal\\content",
            "PortalAdministrator": {
                "UserName": "admin",
                "Email"<EMAIL_ADDRESS>                "Password": "secret",
                "SecurityQuestionIndex": 1,
                "SecurityAnswer": "secretanswer"
            }
        },
        "DataStore": {
            "ContentDirectoryLocation": "E:\\agstore\\relational",
            "EnableFailoverOnPrimaryStop": false,
            "Installer": {
                "Path": "E:\\apps\\content\\Installs\\ArcGIS 10.9.1\\ArcGISDataStore\\Setup.exe",
                "InstallDir": "E:\\apps\\esri\\DataStore"
            }
        },
        "WebAdaptor": {
            "AdminAccessEnabled": true,
            "Installer": {
                "Path": "E:\\apps\\content\\Installs\\ArcGIS 10.9.1\\WebAdaptorIIS\\Setup.exe"
            },
            "AdminAccessEnabled": true
        }
    }
}

Expected Behaviour
Portal should have authorised using json license file and finished initial site configuration.
Actual Behaviour
I get a message indicating the portal site has not been upgraded.
It's a fresh install, but understand these configuration commands are run as part of configuration so looks as though issues relate to license file not applying.
PowerShell DSC resource ArcGIS_Portal  failed to execute Set-TargetResource functionality with error message:
[Error] - Response:-
  {
      "status": "error",
      "messages": ["The portal site has not been upgraded. Please upgrade the site and try again."],
      "isUpgrade": true,
      "resources": ["license"],
      "isPortalLicensed": true,
      "isLicenseFileRequired": true,
      "isUserTypeLicensing": true,
      "version": "10.9.1"
  }
The SendConfigurationApply function did not succeed.

When accessing the portal site via browser it looks as though it could be continued if portal license is applied manually:

Import License File and Continue Portal Upgrade

Steps to Reproduce
Run the following:

Invoke-ArcGISConfiguration E:\administration\configuration\install\BaseDeployment-MultiMachine.json

Important Factoids

Running in an on premise Windows Server 2019.
The ArcGIS Server and Portal license file paths were initially incorrect and failures occurred due to bad file paths. These have been corrected and ArcGIS Server has authorised successfully via the script, but Portal does not want to continue.

References

Hi @dmoiler,
If this is a fresh installation, and not an upgrade, then portal is in a bad state. Portal appears to be stuck in "upgrade" mode, "isUpgrade": true, rather than fresh site mode.
You can confirm this by navigating to https://VMCLIENTMAPS.yourdomain.com:7443/arcgis/home, it likely says "Upgrade" rather than "Create or Join Site".
I recommend uninstalling, deleting the installation directory and content directory, and trying again.
Thanks,
Cameron K.

Hi Cameron,
Thanks for the quick response.
Yes Portal site gives me: Import License File and Continue Portal Upgrade
I will perform the uninstall and try running install once more as suggested and will report back with the outcome.
Thanks,
Dean

Hi Cameron,
After performing uninstall, clearing site content and then install all applications loaded as expected:
Invoke-ArcGISConfiguration E:\administration\configuration\install\BaseDeployment-MultiMachine.json -Mode Uninstall
Invoke-ArcGISConfiguration E:\administration\configuration\install\BaseDeployment-MultiMachine.json

Thanks again for your assistance..
Dean


:bug: virtualworkspace/workspaces: fix generateName

Workspaces with metaname.generateName have empty name. But the VW code depended on workspace.name for roles and bindings and error messages.

/approve