Move Metamodel Content from Part 2 to Part 1

Move Clause 11.4 (Value-Only and idShortPath) and Clause 12.5 (Metadata Objects) from Part 2, V3.0 to Part 1.

newly structured mapping chapter of Part 1 to include different Content Format Types as well:  https://github.com/admin-shell-io/aas-specs/blob/IDTA-01001-3-1_working/documentation/IDTA-01001/modules/ROOT/pages/IDTA-01001_Mappings.adoc.
and
https://github.com/admin-shell-io/aas-specs/blob/IDTA-01001-3-1_working/documentation/IDTA-01001/modules/ROOT/pages/Annex/IDTA-01001_ValueOnlySerializationExample.adoc


Cannot download data

Hi, I cant seem to download the data. I filled the form on the website but nothing happens afterwards. Sorry this question is not related to this repo, but didnt know any other way to point out my issue.

The page should give this link after filling the form.


Fix Oauth2 Authorization header parser

Closes #226
Thanks @toby5box for the bug report.

lgtm, thanks!


[Api-draft] Add spark example

Purpose of this pull request
Add spark example

Check list

[ ] Code changed are covered with tests, or it does not need tests for reason:
[ ] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs

@BenJFan Need to add dependency.


TSC error with Vitest 2.0.5: missing declarations for "chai"

Describe the bug
Using Vitest v2.0.5 gives a type error when TSC typechecks vite.config.ts in strict mode.
node_modules/vitest/dist/chunks/reporters.C_zwCd4j.d.ts:17:23 - error TS7016: Could not find a declaration file for module 'chai'. '/home/projects/stackblitz-starters-xjwz65/node_modules/chai/chai.js' implicitly has an 'any' type.
  Try `npm i --save-dev @types/chai` if it exists or add a new declaration (.d.ts) file containing `declare module 'chai';`

17 import * as chai from 'chai';
                         ~~~~~~

I don't think I should be forced to install some unrelated type package to use Vitest.
Reproduction

https://stackblitz.com/edit/stackblitz-starters-xjwz65
run npx tsc

System Info
[stackblitz output]

  System:
    OS: Linux 5.0 undefined
    CPU: (8) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 0 Bytes / 0 Bytes
    Shell: 1.0 - /bin/jsh
  Binaries:
    Node: 18.20.3 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 10.2.3 - /usr/local/bin/npm
    pnpm: 8.15.6 - /usr/local/bin/pnpm
  npmPackages:
    vite: ^5.4.2 => 5.4.2
    vitest: ^2.0.5 => 2.0.5

Used Package Manager
npm
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

Working around this for now with skipLibCheck: true, but would rather not have to


Debugger error when debugging Flutter Web app with a class with a static function

Steps to Reproduce
When debugging a Flutter Web app, Android Studio's debugger shows an exception and prevents from examining variables when stepping into a function of a class that has a static function that returns an instance of the class, if the class' file name begins with a capital letter.
It's trivial to reproduce.
Create two classes, in two files:
Dbg.dart
class Dbg {
  static Dbg instance() { return Dbg(); }
  void test() {
    int a = 2; // <--- put breakpoint here
  }
}

main.dart
import 'Dbg.dart';

void main() {
  Dbg dbg = Dbg();
  dbg.test();
}

Put a breakpoint on the int a = 2; line, and run in debug mode (on a browser).
Expected results:
Once paused on the breakpoint, the debugger's Variables tool shows the in-scope variables.
Actual results:
Once paused on the breakpoint, the debugger's Variables tool shows an exception text.
Following is a screenshot, showing what it looks like, and the complete exception text:

Note: if I rename Dbg.dart to dbg.dart - the problem goes away.
Note 2: if I comment out the static Dbg instance() { return Dbg(); } line - the problem also goes away.

Logs
flutter analyze just complains about the unused local variable a.
flutter doctor -v:
[✓] Flutter (Channel stable, 2.2.2, on Linux, locale en_US.UTF-8)
    • Flutter version 2.2.2 at /home/user/flutter
    • Framework revision d79295af24 (8 weeks ago), 2021-06-11 08:56:01 -0700
    • Engine revision 91c9fc8fe0
    • Dart version 2.13.3

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /home/user/Android/Sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: /home/user/Downloads/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+0-b944-P17168821)
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/linux#android-setup for more details.

[✓] Chrome - develop for the web
    • CHROME_EXECUTABLE = chromium

[✓] Android Studio (version 4.2)
    • Android Studio at /home/user/Downloads/android-studio
    • Flutter plugin version 58.0.1
    • Dart plugin version 202.8488
    • Java version OpenJDK Runtime Environment (build 11.0.8+0-b944-P17168821)

[✓] IntelliJ IDEA Ultimate Edition (version 2020.2)
    • IntelliJ at /snap/intellij-idea-ultimate/current
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] Connected device (1 available)
    • Chrome (web) • chrome • web-javascript • Chromium 86.0.4240.75 snap

! Doctor found issues in 1 category.

Hi @obeobe, Thanks for filing the issue I could see this working fine on the latest stable channel.

code sample
import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);
  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      _counter++;
    });
    dbg.test();
  }
  Dbg dbg = Dbg();

  @override
  Widget build(BuildContext context) {

    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headline4,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

class Dbg {
  static Dbg instance() { return Dbg(); }
  void test() {
    int a = 2; // <--- put breakpoint here
  }
}

Can you please try upgrading to latest stable channel by running
flutter upgrade
flutter clean

And see if this fixes the issue.
Thank you.

Hi @maheshmnj .
The problem still reproduces for me, after upgrading.
I upgraded per your instructions (to: [✓] Flutter (Channel stable, 2.2.3, on Linux, locale en_US.UTF-8)).
Please note that I am using Chromium (on Ubuntu 20.04). Maybe it's related?
Thanks!

Thanks for the clarification @obeobe, I am able to reproduce the issue on the latest stable and the master channel

code sample
Dbg.dart
class Dbg {
  static Dbg instance() { return Dbg(); }
  void test() {
    int a = 2; // <--- put breakpoint here
  }
}

main.dart
import 'package:counter/Dbg.dart';
import 'package:flutter/material.dart';

void main() {
Dbg dbg = Dbg();
dbg.test();
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);
  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {

    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headline4,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

console log
getObject: Unexpected error from chrome devtools:
text: Uncaught
exception:
  description: TypeError: Cannot read property 'name' of undefined
    at <anonymous>:9:25
    at <anonymous>:43:7
    at Dbg.Dbg.new.test (http://localhost:50974/packages/counter/Dbg.dart.lib.js:17:15)
    at main$ (http://localhost:50974/packages/counter/main.dart.lib.js:466:9)
    at main (http://localhost:50974/web_entrypoint.dart.lib.js:33:29)
    at main.next (<anonymous>)
    at http://localhost:50974/dart_sdk.js:37403:33
    at _RootZone.runUnary (http://localhost:50974/dart_sdk.js:37274:59)
    at _FutureListener.thenAwait.handleValue (http://localhost:50974/dart_sdk.js:32530:29)
    at handleValueCallback (http://localhost:50974/dart_sdk.js:33057:49)
    at Function._propagateToListeners (http://localhost:50974/dart_sdk.js:33095:17)
    at _Future.new.[_completeWithValue] (http://localhost:50974/dart_sdk.js:32943:23)
    at http://localhost:50974/dart_sdk.js:32194:46
    at _RootZone.runUnary (http://localhost:50974/dart_sdk.js:37274:59)
    at _FutureListener.then.handleValue (http://localhost:50974/dart_sdk.js:32530:29)
    at handleValueCallback (http://localhost:50974/dart_sdk.js:33057:49)
    at Function._propagateToListeners (http://localhost:50974/dart_sdk.js:33095:17)
    at _Future.new.[_completeWithValue] (http://localhost:50974/dart_sdk.js:32943:23)
    at async._AsyncCallbackEntry.new.callback (http://localhost:50974/dart_sdk.js:32964:35)
    at Object._microtaskLoop (http://localhost:50974/dart_sdk.js:37526:13)
    at _startMicrotaskLoop (http://localhost:50974/dart_sdk.js:37532:13)
    at http://localhost:50974/dart_sdk.js:33303:9
  type: object
  value: null
attempted JS eval: `    (function() {
         var sdkUtils = require('dart_sdk').dart;
   var library = sdkUtils.getLibrary('package:counter/Dbg.dart');
   if (!library) throw 'cannot find library for package:counter/Dbg.dart';

      var result = {};
      var clazz = library["Dbg$"];
      var descriptor = {
          'name': clazz.name,
          'dartName': sdkUtils.typeName(clazz)
        };
      // TODO(jakemac): static methods once ddc supports them
      var methods = sdkUtils.getMethods(clazz);
      var methodNames = methods ? Object.keys(methods) : [];
      descriptor['methods'] = {};
      for (var name of methodNames) {
        var method = methods[name];
        descriptor['methods'][name] = {
          // TODO(jakemac): how can we get actual const info?
          "isConst": false,
          "isStatic": false,
        }
      }
      // TODO(jakemac): static fields once ddc supports them
      var fields = sdkUtils.getFields(clazz);
      var fieldNames = fields ? Object.keys(fields) : [];
      descriptor['fields'] = {};
      for (var name of fieldNames) {
        var field = fields[name];
        var libraryUri = Object.getOwnPropertySymbols(fields[name]["type"])
        .find(x => x.description == "libraryUri");
        descriptor['fields'][name] = {
          // TODO(jakemac): how can we get actual const info?
          "isConst": false,
          "isFinal": field.isFinal,
          "isStatic": false,
          "classRefName": fields[name]["type"]["name"],
          "classRefDartName": sdkUtils.typeName(fields[name]["type"]),
          "classRefLibraryId" : field["type"][libraryUri],
        }
      }
      return descriptor;
    })()
    `

flutter doctor -v
[✓] Flutter (Channel stable, 2.2.3, on macOS 11.4 20F71 darwin-arm, locale en-GB)
    • Flutter version 2.2.3 at /Users/mahesh/Documents/flutter
    • Framework revision f4abaa0735 (6 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] VS Code (version 1.58.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.25.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 11.4 20F71 darwin-arm
    • Chrome (web)    • chrome • web-javascript • Google Chrome 92.0.4515.131

• No issues found!```

[✓] Flutter (Channel master, 2.5.0-6.0.pre.21, on macOS 11.4 20F71 darwin-arm, locale en-GB)
• Flutter version 2.5.0-6.0.pre.21 at /Users/mahesh/Documents/flutter_master
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 40e8620a27 (4 days ago), 2021-08-06 09:14:50 +0430
• Engine revision 431ac604da
• Dart version 2.14.0 (build 2.14.0-383.0.dev)
[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
• Android SDK at /Users/mahesh/Library/Android/sdk
• Platform android-31, build-tools 31.0.0
• ANDROID_HOME = /Users/mahesh/Library/Android/sdk
• Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 12.5.1, Build version 12E507
• CocoaPods version 1.10.1
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 4.2)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
[✓] VS Code (version 1.58.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.25.0
[✓] Connected device (2 available)
• macOS (desktop) • macos  • darwin-arm64   • macOS 11.4 20F71 darwin-arm
• Chrome (web)    • chrome • web-javascript • Google Chrome 92.0.4515.131
• No issues found!
</details>

Thanks for the clarification @obeobe, I am able to reproduce the issue on the latest stable and the master channel

code sample
Dbg.dart
class Dbg {
  static Dbg instance() { return Dbg(); }
  void test() {
    int a = 2; // <--- put breakpoint here
  }
}

main.dart
import 'package:counter/Dbg.dart';
import 'package:flutter/material.dart';

void main() {
Dbg dbg = Dbg();
dbg.test();
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);
  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {

    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headline4,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

console log
getObject: Unexpected error from chrome devtools:
text: Uncaught
exception:
  description: TypeError: Cannot read property 'name' of undefined
    at <anonymous>:9:25
    at <anonymous>:43:7
    at Dbg.Dbg.new.test (http://localhost:50974/packages/counter/Dbg.dart.lib.js:17:15)
    at main$ (http://localhost:50974/packages/counter/main.dart.lib.js:466:9)
    at main (http://localhost:50974/web_entrypoint.dart.lib.js:33:29)
    at main.next (<anonymous>)
    at http://localhost:50974/dart_sdk.js:37403:33
    at _RootZone.runUnary (http://localhost:50974/dart_sdk.js:37274:59)
    at _FutureListener.thenAwait.handleValue (http://localhost:50974/dart_sdk.js:32530:29)
    at handleValueCallback (http://localhost:50974/dart_sdk.js:33057:49)
    at Function._propagateToListeners (http://localhost:50974/dart_sdk.js:33095:17)
    at _Future.new.[_completeWithValue] (http://localhost:50974/dart_sdk.js:32943:23)
    at http://localhost:50974/dart_sdk.js:32194:46
    at _RootZone.runUnary (http://localhost:50974/dart_sdk.js:37274:59)
    at _FutureListener.then.handleValue (http://localhost:50974/dart_sdk.js:32530:29)
    at handleValueCallback (http://localhost:50974/dart_sdk.js:33057:49)
    at Function._propagateToListeners (http://localhost:50974/dart_sdk.js:33095:17)
    at _Future.new.[_completeWithValue] (http://localhost:50974/dart_sdk.js:32943:23)
    at async._AsyncCallbackEntry.new.callback (http://localhost:50974/dart_sdk.js:32964:35)
    at Object._microtaskLoop (http://localhost:50974/dart_sdk.js:37526:13)
    at _startMicrotaskLoop (http://localhost:50974/dart_sdk.js:37532:13)
    at http://localhost:50974/dart_sdk.js:33303:9
  type: object
  value: null
attempted JS eval: `    (function() {
         var sdkUtils = require('dart_sdk').dart;
   var library = sdkUtils.getLibrary('package:counter/Dbg.dart');
   if (!library) throw 'cannot find library for package:counter/Dbg.dart';

      var result = {};
      var clazz = library["Dbg$"];
      var descriptor = {
          'name': clazz.name,
          'dartName': sdkUtils.typeName(clazz)
        };
      // TODO(jakemac): static methods once ddc supports them
      var methods = sdkUtils.getMethods(clazz);
      var methodNames = methods ? Object.keys(methods) : [];
      descriptor['methods'] = {};
      for (var name of methodNames) {
        var method = methods[name];
        descriptor['methods'][name] = {
          // TODO(jakemac): how can we get actual const info?
          "isConst": false,
          "isStatic": false,
        }
      }
      // TODO(jakemac): static fields once ddc supports them
      var fields = sdkUtils.getFields(clazz);
      var fieldNames = fields ? Object.keys(fields) : [];
      descriptor['fields'] = {};
      for (var name of fieldNames) {
        var field = fields[name];
        var libraryUri = Object.getOwnPropertySymbols(fields[name]["type"])
        .find(x => x.description == "libraryUri");
        descriptor['fields'][name] = {
          // TODO(jakemac): how can we get actual const info?
          "isConst": false,
          "isFinal": field.isFinal,
          "isStatic": false,
          "classRefName": fields[name]["type"]["name"],
          "classRefDartName": sdkUtils.typeName(fields[name]["type"]),
          "classRefLibraryId" : field["type"][libraryUri],
        }
      }
      return descriptor;
    })()
    `

flutter doctor -v
[✓] Flutter (Channel stable, 2.2.3, on macOS 11.4 20F71 darwin-arm, locale en-GB)
    • Flutter version 2.2.3 at /Users/mahesh/Documents/flutter
    • Framework revision f4abaa0735 (6 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)

[✓] VS Code (version 1.58.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.25.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 11.4 20F71 darwin-arm
    • Chrome (web)    • chrome • web-javascript • Google Chrome 92.0.4515.131

• No issues found!```

[✓] Flutter (Channel master, 2.5.0-6.0.pre.21, on macOS 11.4 20F71 darwin-arm, locale en-GB)
• Flutter version 2.5.0-6.0.pre.21 at /Users/mahesh/Documents/flutter_master
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 40e8620a27 (4 days ago), 2021-08-06 09:14:50 +0430
• Engine revision 431ac604da
• Dart version 2.14.0 (build 2.14.0-383.0.dev)
[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
• Android SDK at /Users/mahesh/Library/Android/sdk
• Platform android-31, build-tools 31.0.0
• ANDROID_HOME = /Users/mahesh/Library/Android/sdk
• Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 12.5.1, Build version 12E507
• CocoaPods version 1.10.1
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 4.2)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
[✓] VS Code (version 1.58.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.25.0
[✓] Connected device (2 available)
• macOS (desktop) • macos  • darwin-arm64   • macOS 11.4 20F71 darwin-arm
• Chrome (web)    • chrome • web-javascript • Google Chrome 92.0.4515.131
• No issues found!
</details>

/cc @annagrin

I like to ask you to double check that the file name starting with a capital letter is really required. Does this really not repro when the filename starts with a lowercase latter?

@zanderso yes, it's required. For me it doesn't reproduce if I rename the file to dbg.dart.

/cc @devoncarew

/cc @nshahan I wonder if ddc and dwds disagree on the name of Dbg class in this case.

Can confirm the issue is not reproduced with file named as dbg.dart (with lowercase letters).

@annagrin I think it's likely the case. I'm guessing DDC will use Dbg as the name of the library so the class definition itself will use the name Dbg$. That class definition will be accessible as Dwg.Dwg though and that is now it will be used throughout the application.
This sounds like another issue that will be solved with our work on debugger symbols. The current implementation is probably assuming the name used in the class definition will be the same in Dart and JavaScript. That is more likely to be true when the names follow the guidance in Effective Dart https://dart.dev/guides/language/effective-dart/style#do-name-libraries-and-source-files-using-lowercase_with_underscores. That way the names of the library and class would not clash.


SkillAction: If the skill misses, the battler doesn't lose mana

Thanks! Do you think you could open a PR to fix this? This way you'll appear as a contributor as well

Sure, here is the PR #91


Testing Link on Website is Wrong

The Contributing > Testing link in the lefthand navigation menu links to https://cdn.polyfill.io/v/docs/contributing/testing when it should link to https://cdn.polyfill.io/v2/docs/contributing/testing .
If you point me at the website repo, I'm happy to open a PR.

This is actually already fixed in master - https://github.com/Financial-Times/polyfill-service/commit/0cfd3269df5b19b556fdd1fda0711a96355cad8f#diff-9196b807a42aa88966f3ce475292b219.  Thanks for pointing it out and sorry this got through.  Will be released in 3.8.

Whoops! Sorry about the false alarm.


Check the token before the API call

Hi,
At this moment each call we need to do, for example readAllProfilesByToken, needs us to check if the user token is defined. I think it can be better for your library to check before doing the API call if this token is defined.
This is our code:
if(LoginRadiusSDK.getToken()){
var data = LoginRadiusSDK.readAllprofilesByToken();
if(data){
//check data
}
}
And we think that this solution can be better:
var data = LoginRadiusSDK.readAllprofilesByToken();
if(data){
//check data
}
Thanks.

Hi,
This improvement will be included in the next iteration of the HTML/JS SDK.
Thanks,

This has been released


Source Control on git repository

Type: Bug

clone any git repo
using vscode open the cloned repo
open source control

NOTE: I've already download insider version and same behavior detected, maybe some incompatibility on my environment.
 Unfortunately, it's hard for us to understand what issue you're seeing. Please help us out by providing a screen recording showing exactly what isn't working as expected. While we can work with most standard formats, .gif files are preferred as they are displayed inline on GitHub. You may find https://gifcap.dev helpful as a browser-based gif recording tool.
If the issue depends on keyboard input, you can help us by enabling screencast mode for the recording (Developer: Toggle Screencast Mode in the command palette). Lastly, please attach this file via the GitHub web interface as emailed responses will strip files out from the issue.
Happy coding!

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Add a shrink_to_fit function for the quadratic models.

Application
It will shrink the size of the quadratic models to what is actually required. Though when creating the CQM through serialization we calculate how much memory will be required exactly, there are other phases where that is not possible, it may help in those scenarios if there are later steps to that which allocate memory (so the added memory allocation do not exceed the total system memory). If there are no later steps then it is not required.
Proposed Solution
We just need to call the function shrink_to_fit() for all the vectors contained in the object.
Alternatives Considered
We could have a reserve function, to reserve memory for the individual functions in the quadratic model. This alternative will be more efficient but we will have to insert additional code to calculate the size of the vectors in the different functions. Given the creation of a quadratic model is not one of the hottest functions. shrink_to_fit() is simpler.
Additional Context
There are some functions where we use add/set_quadratic, and we want to keep those functions simple and call shrink_to_fit for the quadratic model objects.

Do you want this just at the C++ level or the Cython level as well? Probably not the Python level...

As far

Do you want this just at the C++ level or the Cython level as well? Probably not the Python level...

For my purpose I only need C++ level one, so that in the function toBQM, I can call it at the end. But as I said it will only matter if we need to allocate more memory after the call to toBQM. Cannot confirm the last statement till I have gone through all the hss code.


cannot find module readble-stream/duplex

node:internal/modules/cjs/loader:942
throw err;
^
Error: Cannot find module 'readable-stream/duplex'

Hi @im-shamil , can you add more information to your issue?
For example:

Node version
Function that you use to send file/...
Code example

node version: v18.3.0
npm version<EMAIL_ADDRESS>code:
const bot = require ("node-telegram-bot-api")```

fixed thankyou

fixed thankyou

can you share how to fix this, thx


[Feature Request] list support

Would it be possible to also support markdown lists such as this:
parent::
- [note1](note1)
friend::
- [note2](note2)
- [note3](note3)
child::
- [note4](note4)
- [note5](note5)
- [note6](note6)

Ive taken to using these as a navigation section for my notes and this would permit a nicer looking list of links when rendered.  Here is an example of what Im currently doing:
**Navigation**

- parent:: [note1](note1)
- peer:: [note2](note2), [note3](note3)
- child:: [note4](note4),[note5](note5), [note6](note6)

This works, ok, but gets unweieldy when there are many links to contend with.

A good idea.  I have same challenge when build project heirarchies


github.com/DataDog/zstd cc1.exe: sorry, unimplemented: 64-bit mode not compiled in

Bug description
Briefly describe the bug.
When we type the command
go build -o sim
I get
github.com/DataDog/zstd cc1.exe: sorry, unimplemented: 64-bit mode not compiled in
Go version
Which version of Go are you running?

Operating system: Windows 7 home
RAM:8GB
Cores: 4
Device: i3

Steps To reproduce the bug
Explain how the maintainer can reproduce the bug.

install go.1.15.3 on windows
Installing and Updating Cygwin for 64-bit versions of Windows
go build -o sim step gives github.com/DataDog/zstd cc1.exe: sorry, unimplemented: 64-bit mode not compiled in

Expected behaviour
No errors
Actual behaviour
github.com/DataDog/zstd cc1.exe: sorry, unimplemented: 64-bit mode not compiled in
Errors
github.com/DataDog/zstd cc1.exe: sorry, unimplemented: 64-bit mode not compiled in

Got the same bug, go version
go version go1.18.5 windows/amd64
Windows 11, RAM 32 GB
github.com/DataDog/zstd
cc1: sorry, unimplemented: 64-bit mode not compiled in


Add japanese translation for 22.0 release

Lightly tested ACK e4a78adc849281eb81d2ee3612127c01573ab91c Thanks!

see https://github.com/bitcoin-core/bitcoincore.org/pull/810#issuecomment-1014365302


Exit with code 1 on failure

Getting an error code on failure is pretty important when running migrations from a script.
For example, I would like to apply migrations in an init script in the official postgresql docker image in our CI, and also want to automate it in our deploy scripts. In both of these use-cases detecting that the migration failed is necessary.

Hey, that's a good catch. Thank you for PR.
Quick Question - Instead of add "return" command everywhere isn't better to add in the method handleError the exit(1)?

The Symphony Command class wants the exit status returned from execute so I thought I'd better stick to that. No idea if it really matters though. 😄

All right, it is good to me. We need to handle the parent::execute though.

Not quite sure I understand what you mean with "handle the parent::execute"?

Something like this:
$result = parent::execute($input, $output);
if ($result == 0) {
   // Do what have to do
}
return $result;

Ok, updated. WDYT?
Didn't have time to even run this yet though.

I am merging to branch 4.1.1 while I am doing some tests.
Meanwhile, you can download the version 4.1.1.x-dev to use this code.


Module 4.1 Assignment

👋 Hi, you are missing the Module 4.1 Assignment. Please post it here in this thread.


[WIP] 5.2.3

Features

[x] Cleanup
[x] Better jsdoc for docs
[x] Command classes
[x] Cooldown bypass for app owner/team
[x] Cooldown fix
[x] Typings for command classes
[x] Event classes
[x] Typings for event classes
[x] Workflows (auto checks for PRs)
[x] Min node version in package.json
[x] GPayload
[x] GPayload typings
[x] Sending only attachments in message
[x] Sending only embeds in message
[x] Back Structures with djs v13
[x] Removed guild.prefix/language (use getCommandPrefix(), getLanguage())
[x] Added .setDisabled() to MessageSelectMenu
[x] Support ws(raw) events
[x] client.events -> client.gevents
[x] interaction.think() in slash cmd
[x] interaction.createdTimestamp
[x] GInteraction typings
[x] GInteraction jsdoc
[x] GInteraction
[x] GPayload test
[x] Normal arguments
[x] expectedArgs => args
[x] SlashCommand.STRING => ArgumentType.STRING
[x] GEvents to @gcommands/events
[x] Case sensitive commands
[x] Embed attachments fix
[x] Major fixes
[x] Final Testing

Website

[x] Guide gcommands.js.org/guide
[x] Docs gcommands.js.org/docs
[x] gcommands.js.org => gcommands.js.org/docs(Auto redirect to docs)

Status

[x] These changes have been tested and formatted properly.
[x] This PR introduces some Breaking changes.

Hyro, on 240 line in GMessage.js there is an conflicting expression, replace updater.checkDjsVersion("13") with ifDjsV13!

resolved

https://github.com/Garlic-Team/GCommands/issues/39

Preparation for normal arguments.
args: [
 // etc (same as slash args) https://gcommands.js.org/guide/guide/beginner/arguments.html#without-slash-group
]

I think everything is corect.


Projects as resources

After putting the Resources page to use in several events, the use and shortcomings of the design were better appraised. It is clearly useful to better describe the progress of projects, but having a separate model for a Resource is both too constraining and an unnecessary overhead. Instead, we will come back in #249 to using a Project to define tools and resources for other projects, using the full flexibility of the systems we have in place.
In this Pull Request:

Resource models and related code and table references are removed, however the migrations do not delete your data.
The progress outline and content of the Resource page is now in an Information page.
A new configuration file - stages.yaml - defines project progress.
When writing a Post, the user is shown an assessment, and the project is promoted on a completed submission.
Only administrators in the backend can now arbitrarily change a Project's progress level.
Project-related views were refactored into a separate project.py blueprint.

Codecov Report

Merging #251 (7b1f1cd) into main (8ee1d42) will decrease coverage by 0.65%.
The diff coverage is 37.36%.

@@            Coverage Diff             @@
##             main     #251      +/-   ##
==========================================
- Coverage   47.96%   47.30%   -0.66%
==========================================
  Files          27       28       +1
  Lines        2627     2490     -137
==========================================
- Hits         1260     1178      -82
+ Misses       1367     1312      -55

Impacted Files
Coverage Δ

dribdat/apifetch.py
16.47% <ø> (ø)

dribdat/public/api.py
30.04% <0.00%> (ø)

dribdat/settings.py
98.63% <ø> (-0.02%)
:arrow_down:

dribdat/user/__init__.py
100.00% <ø> (ø)

dribdat/public/project.py
18.71% <18.71%> (ø)

dribdat/aggregation.py
10.43% <21.42%> (-14.39%)
:arrow_down:

dribdat/user/constants.py
68.18% <42.10%> (-22.30%)
:arrow_down:

dribdat/onebox.py
52.94% <50.00%> (ø)

dribdat/public/views.py
59.09% <57.89%> (+28.93%)
:arrow_up:

dribdat/user/models.py
59.45% <80.00%> (+0.76%)
:arrow_up:

... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8ee1d42...7b1f1cd. Read the comment docs.

Deployed for staging at https://meta.dribdat.cc/event/1/instructions

What's still missing here is a strategy to link projects to each other in a way that a "Built With" query will be possible. I am thinking about a Components table, where users can add projects as references. This could also just be a link. The user would need to be able to edit and remove the references. The main difference to the way it was before, is that we would link ad hoc, and not just as an Activity update. However, maybe just links in the pitch or read me would suffice. But we would still need to annotate them. The question comes down to how much utility for users this function really has.

This has been extensively tested and staged, and the PR has been rather beaten to death. Release time!


Complete interview with sender Genabio

cc: @CnoteNava
I just created this ticket after finishing ticket 7236 (the interview was scheduled). I'm hoping the info in the epic is sufficient but please let me know how you'd like this type of ticket to be written.


Objective Level [Examples]

Duplicate ticket, see #323

Duplicate ticket, see https://github.com/cdisc-org/DDF-RA/issues/323


helmnet: Removed useless imports

I found some unused imports confusing so I removed them
tested by the successful start of the training and evaluation

Looks great @sio13 , thanks for the PR :)


[regression-test]Add ssb sf0.1 test under unique table with zstd

Proposed changes
Issue Number: close #xxx
Problem Summary:
Add ssb sf0.1 test under unique table with zstd
Checklist(Required)

Does it affect the original behavior:

[ ] Yes
[x] No
[ ] I don't know

Has unit tests been added:

[ ] Yes
[x] No
[ ] No Need

Has document been added or modified:

[ ] Yes
[ ] No
[x] No Need

Does it need to update dependencies:

[ ] Yes
[x] No

Are there any changes that cannot be rolled back:

[ ] Yes
[x] No

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

use title like this enhancement add ssb sf0.1 test under unique table with zstd


3.6.0 blank screen or crash on launch (3.5.0 still works)

OS: Windows 10
Issue Descriptions: Since updating to 3.6.0, my client launches and ends up at either a blank black screen (after the spinner animation stops and disappears), or crashes entirely. The 3.5.0 version is still present alongside 3.6.0, and launching the executable from the 3.5.0 folder instead provides a working client.
(Side note: your bug template says to attach DEBUG information ZIP and provides a FAQ URL for instructions, but the word "debug" appears nowhere at that URL)
Steps to Reproduce: Launch 3.6.0 client, wait for spinner animation to cease, observe either crash or blank window.

Known issue that I am tracking upstream.  It's a memory thing, possibly language specific.  Closing as upstream but what is your system language?

English (US)

@brendonrapp Can you try the most recent CI build from AppVeyor?

I installed the most recent successful build, and there is a slight change of behavior, in that I now get the "Loading Music Library" loading bar after the spinner animation goes away, but after that, I get the same behavior as before: either a black screen or a full crash.


config: add reana-client-go to CLIENT repositories

Useful for commands such as reana-dev git-fetch -c CLIENT etc.
Modifies reana-dev client-install accordingly.

Codecov Report

Merging #676 (7e82ad3) into master (f056bb6) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #676   +/-   ##
=======================================
  Coverage   17.53%   17.53%
=======================================
  Files          24       24
  Lines        2110     2110
=======================================
  Hits          370      370
  Misses       1740     1740

Impacted Files
Coverage Δ

reana/reana_dev/client.py
0.00% <ø> (ø)


Does not work when Upgraded to React v16 and React-router v4.

I have been working on a new mern stack insipred from mern-starter and react-universally. The stable version is still on 15 though. But i have a branch for react 16 with react router 4, the only thing that is not working is the tests because react-addons-test-utils needs to be changed.
The url for it is https://github.com/Ballpin/mern-starter2

Duplicate of #330 and we'll thoroughly test it before we release it.


Full screen problems on 10.6

GLFW 3.0.4 doesn't switch to fullscreen properly on OS X 10.6, and neither does GLFW 3.1 (i hoped that this update fixed that bug).
GLFW 3.0.4: i see a fullscreen window, but it remains black. OpenGL gives no errors.
GLFW 3.1: i don't see a window at all. OpenGL gives no errors.
Please look into this. GLFW is switching to fullscreen just fine on my newer OS X machines and on linux.

Do the examples work?

Also, does GLFW emit any errors?

nothing works as it should...

It is possibly helpful to you to say that the fullscreen mode worked well on OS X 10.6 with the older GLFW 2 series.

It is, but not sufficiently to know what the problem is. The full screen code was completely rewritten for 3.1.

Does GLFW 3.0 work on 10.6?

Yes it does, well i the parts that my applications use do indeed work on OS
X 10.6. I figure you don't have a machine at your disposal to test and
reproduce this bug?

Yes, unfortunately.

Once i find the time, i'll dive into the GLFW source and see if i can get
it to work well on a variety of OS X versions. I do have 10.6, 10.7, and
10.10 machines at my disposal. There are issues with mouse hiding as well
(OS X 10.7, maybe 10.6 too). As i said before, the GLFW 2 fullscreen code
worked well.

Hi @notnot,
Is there a specific reason holding you back at Mac OS X 10.6 instead of upgrading to the latest OS X 10.10? OS X version 10.10 and all later updates are free, and include many under-the-hood as well as usability improvements. Is your Mac an older model that 10.10 does not support?

I do have all kinds of OS X versions running myself, including the 'latest
and greatest', i do have a few machine that are too old to be updated to
10.10. My software is used by other people too, and should thus run on
whatever machines they have. It is more a question of principle: i want to
support a broad as possible range of machines. It is technically possible
to have perfect fullscreen behavior on OS X 10.6 (see GLFW 2), so why not
have it in GLFW 3...
On 1 May 2015 at 06:18, Dmitri Shuralyov<EMAIL_ADDRESS>wrote:

Hi @notnot https://github.com/notnot,
Is there a specific reason holding you back at Mac OS X 10.6 instead of
upgrading to the latest OS X 10.10? OS X version 10.10 and all later
updates are free, and include many under-the-hood as well as usability
improvements. Is your Mac an older model that 10.10 does not support?
—
Reply to this email directly or view it on GitHub
https://github.com/glfw/glfw/issues/448#issuecomment-98040287.

I see, that's a fair point. (Unfortunately I cannot assist with fixing an issue for a non-latest OS X version.)

It is technically possible to have perfect fullscreen behavior on OS X 10.6 (see GLFW 2), so why not have it in GLFW 3...

There is a limited amount of time available to work on GLFW and there are far more users running 10.7 or later than are running 10.6. As long as there are bugs that affect newer versions, those get priority. I'll gladly accept tested patches, however.
If this isn't resolved by the time 3.2 is done, I'll remove compatibility with 10.6 before release.

Sad but understandable. It isn't important enough for me at the moment to
justify development time on it either...

Also if this is added in its current form, 10.6 won't work as well... Unless you build on a newer SDK and target 10.6: 8d2bc5314f38f2cd636da1025f1e68137e6f165d is added to master

@jaredjones Yep. I'll fix that before release if this gets resolved.

Hey, I'm able to get the gears example to go fullscreen on OSX 10.6.8. Is that version too new? I had to change cocoa_window.m a bit to get it to compile, but no other changes were needed.

Great! Will the changes you made be incorporated into glfw? If not, i'd
like to have a copy of your cocoa_window.m to patch my own old 10.6.8
laptop.
On 19 October 2015 at 03:05, AngeloYazar<EMAIL_ADDRESS>wrote:

Hey, I'm able to get the gears example to go fullscreen on OSX 10.6.8. Is
that version too new? I had to change cocoa_window.m a bit to get it to
compile, but no other changes were needed.
—
Reply to this email directly or view it on GitHub
https://github.com/glfw/glfw/issues/448#issuecomment-149071563.

@AngeloYazar It's the best version.
@notnot If the changes allow GLFW to function properly on 10.6 then absolutely!

@notnot I'm not sure if it will fix the problem for you, since I really didn't fix anything. I just made sure it would compile by adding back in all the #if MAC_OS_X_VERSION_MAX_ALLOWED >= 1070 stuff and making a convertRectToBacking function. Try building with this file to see if it works for you: https://gist.github.com/AngeloYazar/1210af2206f8dbf63b51
Opening the examples to fullscreen worked just by adding glfwGetPrimaryMonitor() to glfwCreateWindow. Was that the original error? Or was there a different issue?
@elmindreda agreed!

@angelo, i don't have access to a 10.6.8 machine at the moment, so i can't
try it.
On 19 October 2015 at 21:25, AngeloYazar<EMAIL_ADDRESS>wrote:

@notnot https://github.com/notnot I'm not sure if it will fix the
problem for you, since I really didn't fix anything. I just made sure it
would compile by adding back in all the #if MAC_OS_X_VERSION_MAX_ALLOWED >=
1070 stuff and making a convertRectToBacking function. Try building with
this file to see if it works for you:
https://gist.github.com/AngeloYazar/1210af2206f8dbf63b51
Opening the examples to fullscreen worked just by adding
glfwGetPrimaryMonitor() to glfwCreateWindow. Was that the original error?
Or was there a different issue?
@elmindreda https://github.com/elmindreda agreed!
—
Reply to this email directly or view it on GitHub
https://github.com/glfw/glfw/issues/448#issuecomment-149319458.

@notnot Do you have some example code that was failing? I can try to run it on my machine. Also, do you remember if it was related to switching between windowed and fullscreen?

the example fullscreen code was failing: no window at all was opened. and
all my own programs failed (no window)

Okay great, we just need someone to confirm that the code is working.

Adding this to 3.2 so I won't forget to close it if it hasn't been resolved by then.


Travis should test the network install

Travis should test that it can stack/cabal install the package from the network.

Nope.


Remove old docs files

This PR removes old docs files associated with the Sphinx docs and the OG-USA guide pdf file which has source file in another repo.  These have recently been replaced with the OG-USA Jupyter-Book documentation.

At some point, will need to take down the readthedocs page: https://og-usa.readthedocs.io as it will no longer update after this PR.  I'm just not sure how to do that... not seeing option to delete a project from my account.

Codecov Report

Merging #646 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #646   +/-   ##
=======================================
  Coverage   84.30%   84.30%
=======================================
  Files          47       47
  Lines        6959     6959
=======================================
  Hits         5867     5867
  Misses       1092     1092

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a486425...088c389. Read the comment docs.

@jdebacker. I would leave OGUSAdoc.pdf in the repository for now. There is still a lot of work to get the Jupyter Book documentation to look like the LaTeX-produced PDF. For example, I used the LaTeX-produced PDF to clean up the demographics.md documentation file in PR #644. There is a lot more of that still to do.
Regarding deleting the readthedocs site, I found this StackOverflow link that seems like it is the answer.

Ok - I thought things would be in good enough shape after PR #644.
We can let this sit until we're ready to remove the PDF.
I'll take down the readthedocs when this PR is merged.  Thanks for the link.

@jdebacker. I think we should move ahead with this PR and remove the readthedocs files and site. I just think we should keep the OGUSAdoc.pdf in place until the Jupyter Book documentation is closer. Of particular value are tables, figures, equations, and definition-type items. Let me know when you think this is ready.

@rickecon Latest commit adds back the OGUSAdoc.pdf.


Reserve large fixed-size flat address spaces as files

The EVE graphics controller can render ASTC textures straight from SPI flash. We plan to integrate LittleFS to enable easy asset replacement. One requirement is that the ASTC texture is located in one contiguous area. The current file allocation strategy does not permit this due to the block header in files larger than one block.
To achieve our requirements, this PR adds an additional file record type LFS_TYPE_FLATSTRUCT flagged by LFS_F_FLAT. This record simply stores the first block number and the file size. During block traversal, all the blocks starting from the first block until the last are simply visited (without accessing storage). This allows us to effectively reserve flat address spaces in flash within the filesystem.
A function lfs_file_reserve is added to turn a file into a reserved flat address space. Calling this function will discard the file and attempt to allocate the requested size as a contiguous block. Specifying size 0 will discard the reservation and make the file usable as a regular file again. As we are accessing the reserved storage areas directly from the GPU, no implementation of read nor write is provided. The file is treated as an opaque storage area. Call lfs_file_reserved to get the first block, or to check whether a file is a reserved flat area.
To allocate the contiguous blocks, two strategies are used depending on the size of the file.
The first allocation strategy is to simply allocate blocks, and reset the allocation starting point whenever an expected block is skipped over, until the expected number of blocks has been allocated sequentially. The strategy is used when the file size is smaller than the lookahead cache, and is aborted when more blocks than are contained in the cache have been attempted.
The second allocation strategy allocates one block in the normal fashion. Then traverses the filesystem and advances the allocation starting point anytime a block is found that collides with the expected allocation space, past the colliding block. The traversal is repeated until a complete traversal is done with no collisions, and aborted if the attempted starting point has looped around the address space.
When calling the reserve function on an existing already reserved file, a new allocation is made. As the updated file record is only committed to the storage upon closing the file, and when no error is flagged, this mechanism seems suitable for atomically upgrading large assets on flash.
Usage
Writing a large file
lfs_file_open(&lfs, &file, filename, LFS_O_WRONLY | LFS_O_CREAT);
err = lfs_file_reserve(&lfs, &file, size, 0);

if (!err) {
  // Erase, write, and verify from GPU RAM directly to SPI flash
  lfs_block_t block;
  err = lfs_file_reserved(&lfs, &file, &block);
  if (!err) {
    EVE_CoCmd_flashUpdate(phost, EVE_FLASH_BLOB_SIZE + (block * lfs->cfg->block_size), gpuSrcAddr, size);
    if (!EVE_Cmd_waitFlush(phost)) {
      // Do not commit the new reservation in case of write error
      lfs_file_reserve(&lfs, &file, 0, LFS_R_ERRED);
    }
  }
}

if (err) {
  // Failed to reserve, optionally remove reservation and fallback to normal file behavior
  lfs_file_reserve(&lfs, &file, 0, 0);
  // ... lfs_file_write(...)
}

lfs_file_close(&lfs, &file);

Reading a large file
lfs_file_open(&lfs, &file, filename, LFS_O_RDONLY);

// Read from SPI flash directly to GPU RAM
lfs_block_t block;
if (lfs_file_reserved(&lfs, &file, &block) == LFS_ERR_OK) {
  EVE_CoCmd_flashRead(phost, gpuDstAddr, EVE_FLASH_BLOB_SIZE + (block * lfs->cfg->block_size), size);
  // (Or, alternatively, use the flash address to render ASTC textures directly from SPI flash)
} else {
  // Not a reserved flat file, optionally fallback to normal behavior and read as a file instead
  // ... lfs_file_read(...)
}

lfs_file_close(&lfs, &file);

Flags

LFS_R_ERRED: Flag a writing error so the reservation will not be committed on close
LFS_R_GOBBLE: Force to only use the lookahead allocation strategy, for testing purposes
LFS_R_FRONT: Use only the traversal allocation strategy, start allocating in front, for tooling flash images
LFS_R_TRUNCATE: Attempt to use truncate to shrink or extend the existing reservation, otherwise allocate normally
LFS_R_OVERWRITE: Allow overwriting the previous reservation, useful with LFS_R_FRONT and LFS_R_COPY to repack
LFS_R_COPY: Copy existing blocks from the previous area to the new one, no padding

No idea why the automated test is failing. This shouldn't affect anything existing since all the behavior is behind the file type flags. Unless there's some corrupt data existing in the test that happens to hit those flags?
====== results ======
tests/test_exhaustion.toml:353:failure: test_exhaustion#5#1 (LFS_BLOCK_CYCLES=5) failed
tests/test_exhaustion.toml:[44](https://github.com/littlefs-project/littlefs/runs/6683251429?check_suite_focus=true#step:11:45)9:warn: max wear: 19 cycles
tests/test_exhaustion.toml:[45](https://github.com/littlefs-project/littlefs/runs/6683251429?check_suite_focus=true#step:11:46)0:warn: avg wear: 0 cycles
tests/test_exhaustion.toml:451:warn: min wear: 0 cycles
tests/test_exhaustion.toml:[46](https://github.com/littlefs-project/littlefs/runs/6683251429?check_suite_focus=true#step:11:47)2:warn: std dev^2: 8
tests/test_exhaustion.toml:463:assert: assert failed with 8, expected lt 8
tests/test_exhaustion.toml:463:assert: assert failed with 8, expected lt 8
    assert(dev2 < 8);

tests passed 873/874 (99.9%)
tests failed 1/874 (0.1%)

This feature would also be interesting to MicroPython, where we need a filesystem which can hold contiguous (read-only) files that can be memory mapped.  Eg they can contain resources, bytecode and machine code which can be executed in place (XIP).  Essentially we would use littlefs (with this PR) as a container format to store a set of assets in the layout of a filesystem.

====== results ======
tests/test_exhaustion.toml:353:failure: test_exhaustion#5#1 (LFS_BLOCK_CYCLES=5) failed
tests/test_exhaustion.toml:[44](https://github.com/littlefs-project/littlefs/runs/6683251429?check_suite_focus=true#step:11:45)9:warn: max wear: 19 cycles
tests/test_exhaustion.toml:[45](https://github.com/littlefs-project/littlefs/runs/6683251429?check_suite_focus=true#step:11:46)0:warn: avg wear: 0 cycles
tests/test_exhaustion.toml:451:warn: min wear: 0 cycles
tests/test_exhaustion.toml:[46](https://github.com/littlefs-project/littlefs/runs/6683251429?check_suite_focus=true#step:11:47)2:warn: std dev^2: 8
tests/test_exhaustion.toml:463:assert: assert failed with 8, expected lt 8
tests/test_exhaustion.toml:463:assert: assert failed with 8, expected lt 8
    assert(dev2 < 8);

Huh. Somehow incrementing the LFS_DISK_VERSION is causing this test to fail.

Perhaps it may be interesting to separate the size into size and capacity, using reserve to set the capacity, truncate to set the size, and then adding support for regular read and write operations as well.

Happy with this patch as it is now. Does the job so far in testing for our use case.

Hi @kaetemi, sorry about such a large delay in getting to this. This is really great work! Working around the lookahead buffer for large files is especially clever.
A bit of unfortunate timing, but I've been looking into some larger changes to the file data-structure for performance reasons. In particular I'm planning to move from the existing CTZ skip-list to a more traditional B-tree. Given that a B-tree would not require invasive pointers in the data-blocks, it seems a B-tree would allow you to create the lfs_file_reserve without needing a new data-structure. This would have the additional benefit of flat files appearing as normal files on littlefs drivers that don't support flat files.
Thoughts?

It's worth highlighting that this is a ~11% code size increase (15722 B -> 17462 B on thumb). This isn't a deal-breaker, just that we should either put this behind #ifdefs to let users not compile it in, or at least put an additional measurement into CI to keep track of the minimal littlefs code size.

Huh. Somehow incrementing the LFS_DISK_VERSION is causing this test to fail on the powerpc and mips nand test targets.

Ah, that test measures wear distribution and asserts that the standard deviation is within reasonable values. I've also noticed it's flaky on NAND, it writes a fixed amount of data and this currently isn't enough for inter-block wear-leveling to kick in with NAND's large block sizes. It's up to chance if the same block gets selected twice for metadata which will make it fail the check.
I wouldn't worry about it, we can disable the test for NAND for now and create an issue to make the test more robust (which probably just means throwing more data at it).

Given that a B-tree would not require invasive pointers in the data-blocks, it seems a B-tree would allow you to create the lfs_file_reserve without needing a new data-structure. This would have the additional benefit of flat files appearing as normal files on littlefs drivers that don't support flat files.

That sounds interesting. Would it be better in that case to mark the relevant files with a flag to specify them as "flat", or should it just run through the B-tree to check if the file qualifies whenever it is needed?

It's worth highlighting that this is a ~11% code size increase (15722 B -> 17462 B on thumb). This isn't a deal-breaker, just that we should either put this behind #ifdefs to let users not compile it in, or at least put an additional measurement into CI to keep track of the minimal littlefs code size.

Yes, agreed. Better to put it behind a macro so it can be taken out when not needed.

That sounds interesting. Would it be better in that case to mark the relevant files with a flag to specify them as "flat", or should it just run through the B-tree to check if the file qualifies whenever it is needed? (Performance / complexity trade off when opening the files.)

I think a flag would be worth it in this case, but it depends on your use case. If you're going to read the whole file anyways it may not cost anything extra to scan the whole B-tree.
In this hypothetical you could at least store if a file is contiguous in a custom attribute, without requiring in-filesystem changes.
That being said I've been considering including offset+length information in each pointer in the B-tree. This helps take advantage of unused-erased parts of the inner-nodes in the B-tree and as a side effect would let you encode an entire flat file as a single "pointer". Though the intention would be that LittleFS itself remains block-based for allocation, etc.
Sorry I don't have more information, this is all just in the untested design phase at the moment.

I think a flag would be worth it in this case, but it depends on your use case. If you're going to read the whole file anyways it may not cost anything extra to scan the whole B-tree.

A flag would be ideal then. In our case the flat files are generally not read by the microcontroller, they're used directly by the graphics chip. And preferably bootup times of the application should be fast. :)
As long as we can create files that effectively just reserve a flat address space that can be accessed directly outside of the filesystem library, it suits our requirements.

Effectively being able to mmap a file is a great benefit, since it would allow to save a ton of precious RAM (many code perform a copy from the file in flash to RAM for parsing the data in place, a mmap would skip this). Is there any progress on this ?

mirroring @X-Ryl669 , I'd find it very useful in one of my projects (Adding LittleFS to the Game & Watch homebrew scene). We memory-map ROM files, allowing us to run games that we couldn't ever fit in RAM. Currently ROMs are just compiled into the firmware, allowing this memory mapping. But, it would be so much nicer if these were part of the filesystem (and still be memory mapped).

@BrianPugh You can have a look at FrogFS that supports mmap file via its faccess function. It's readonly, but it would fit perfectly for your use case (and you can update it OTA, so it's a very good plus here).

thanks for the reference! I'd say FrogFS almost fits my usecases, but has the drawbacks:

I then have to juggle 2 filesystems (I still need littlefs for writing game-saves).
A super nice-to-have is the ability to incrementally add/delete files (i.e. a normal filesystem). Users commonly have 64MB flash chips which takes ~15 minutes to flash, making adding/removing games less of a casual process.

These are not absolute dealbreakers, but definitely doesn't make it seem like the obvious choice. A nice part if we were to use FrogFS is that it separates out the ROM data from the compilation process, allowing us to make pre-compiled firmware releases (and thus, simplifying the user-experience).

If littlefs had a way to set the "don't split a file below 64KB" mode

If you want bad ideas, you could set the block_size to 64KiB. Then 64KiB files would always be contiguous.

Sorry, I've holding off on updating this issue until things are more actionable.
This is currently blocked by me behind B-tree support, as with B-trees we should be able to provide contiguous files without an additional file structure. And the fewer file structures we have floating around, the less code we need for backwards compatibility.
This is work is mostly complete, but mostly complete is still a bit far from ready to stabilize.
Contiguous files are also messing with plans for future features in complicated and messy ways. I think the general issue is the filesystem wants control of data location for filesystem purposes and contiguous files limit that:

Error-correction - Contiguous files and byte-level error correction are basically incompatible. Most schemes involve interspersing error-correcting codes into the data, which violates the contiguity of contiguous files.

Data-level redundancy - Data redundancy via parity blocks is more doable but presents problems when trying to repair data blocks. Like any other write, repairs need to be copy-on-write, which means we need to copy blocks somewhere else. But if we copy a block in a contiguous file, this violates contiguity. We could copy-on-write the entire contiguous file, but this adds code cost and is more likely to fail due to space/fragmentation issues.
My current thinking is that contiguous files would be unavailable for data redundancy. They could still work with data-level checksumming though, leaving it up to the user to handle errors.

Static wear-leveling - Again, the filesystem is copy-on-write, and this conflicts with contiguous files. We would need extra code to handle the special case of contiguous files, and is more likely to fail due to space/fragmentation issues.
There is also the issue of static wear-leveling invalidating external references to the contiguous data.
So contiguous files would probably also be unavailable for any potential static wear-leveling.
This effectively makes contiguous files more like a hole in the filesystem, rather than actual files.

All of these special cases invalidate ealier comments of mine. Contiguous files are going to need a flag, and must be a recognized by littlefs to at least avoid corrupting their contiguousness.

For a while I was considering tying contiguous files to the future-planned optional on-disk block-map.
So instead of allowing filesystem-maintained contiguous files, allow users to reserve contiguous blocks that are tracked as reserved in the block-map.
Users could then store info about the allocated blocks in a file if they wanted to the file to appear in the filesystem tree.
This would add more work to users, but it would make it clear contiguous files don't benefit from other filesystem features.
But then I realized this creates a unavoidable race condition. There is no way for users to avoid leaking storage if power is lost between the block allocation and the writing of tracking file to disk.
So it seems contiguous files need to be implemented in the filesystem in order for the allocation + tracking to be atomic.

So the best plan to me still seems to be:

Add an API to allow requesting a contiguous file of a given size, using @kaetemi's scheme for contiguous allocation.
Mark contiguous files as contiguous on-disk so the filesystem knows to work around them
Allow access to the physical address for things like XIP, mmap, etc.

But this all depends on B-tree support, so I don't think it's actionable yet.
There's some other questions, like should we even support write on contiguous files vs letting users call the bd_erase/bd_prog directly when they have the physical address, but figuring out the details can probably wait until after B-trees...

And it's not really that this feature has low-priority, I realize this is useful for users, just that other features have higher priority. Sorry about that.

Mark contiguous files as contiguous on-disk so the filesystem knows to work around them

Actually the on-disk information you need is more like "pinned". That a file's data blocks should not be moved.
That the file is contiguous is only needed at allocation time.

With the recent addition of WAMR for ESP32, you can now run XIP's WASM binary from flash, which you could update dynamically without having to reboot, a bit like a true OS.

It think it is worth mentioning that if you only have one binary, you can always partition your disk to store that binary outside of the filesystem while still storing its metadata in the filesystem. The partitioning could be an actual partition table or hardcoded.
You could extend this to a sort of storage pool for contiguous files, though this path could end up hitting rough edges.

For our use case writing/updating contiguous files is not necessary. We would be happy with a purely read-only filesystem (after it's created), although the ability to append/add new files to this "read only" filesystem would be nice.

I'm curious, if this is a read-only image, could you not allocate blocks for the contiguous files you need from the end before creating the filesystem, and store "fake" files in the filesystem that describe where they live on disk? This is pretty much equivalent to what you would get with contiguous files, since contiguous files also can't benefit from inlining.
(Maybe contiguous files could be inlined, but that would force their metadata block to become sort of frozen, which would get really messy with littlefs's design).
This would also allow your contiguous files to share blocks, which unfortunately I don't think will be possible with contiguous files in littlefs.
From what I understand the benefit of supporting contiguous files inside the filesystem is specifically to allow rewriting and dynamic allocation from littlefs's pool of blocks.

if this is a read-only image, could you not allocate blocks for the contiguous files you need from the end before creating the filesystem, and store "fake" files in the filesystem that describe where they live on disk?

I've actually proposed a similar thing for my project, but was waiting a little bit to see how this PR would evolve. I'd be willing to volunteer a small C/python project that does this. There are a few complications, but nothing unsolvable.
The primary issue with this, is that we've essentially partitioned the storage, which could be an inefficient use. E.g. How do I balance how much of my flash I should allocate to LittleFS vs this secondary contiguous filesystem? This is especially imbalanced for my use-case (game ROMs in contiguous storage, compressed savestates and other smaller files in LittleFS). It would be much nicer if they could all share the same partition.

While thinking about this, I think there are numerous intermediate steps that can be done that would closely fit the need:

First, the need for contiguous space is mainly required for reading. We could live without contiguous write support, IMHO.
On my system, there's a MMU unit, so it's possible to map the flash address to the µc unit's address space. I think the need for this feature is mainly for XIP or large data mapping. In the former case, there's a MMU somehow on the system.
So, given the fact there's a MMU, the only requirement becomes: have a "virtual" block that's the same size as the MMU unit, because using the MMU can make any block appear at linear space.
In a B-Tree like structure, I think it's possible to have multiple type of nodes, for different "block" sizes, without too much impact on the logic of the code (this requires tests for the node type where size is computed)
So, it might be possible to keep the new B-Tree structure to deal with some "super" 64kB blocks that can't be split anymore, yet, being able to move them
In the end, a mapping function could be added (via a callback). It'll be called for each block address in logical data index so that the MMU unit can be programmed to make a virtual, contiguous, address space.
Additionally, a "memory mapped" bit could be added to the file or B-Tree node, so when this bit is set, no move of these blocks is allowed. They only need to be in RAM (not flash), since after reset, the mapping is gone. These bits are removed in the unmap function.
In the previous requirement can't be met, maybe a callback "force unmap" is possible too, to force the CPU to unmap the block in case Littlefs needs to move them. This could allow the CPU to retry mapping the files once they are changed on disk.

The primary issue with this, is that we've essentially partitioned the storage, which could be an inefficient use. E.g. How do I balance how much of my flash I should allocate to LittleFS vs this secondary contiguous filesystem?

Fortunately this goes away when your image is read-only. At creation time you know exactly how much storage you need for the contiguous files, so that's how much you allocate and you can give LittleFS the rest.
But I do see where in-filesystem support is useful for dynamic contiguous files. Unfortunately navigating how to fit it into the filesystem gets messy.

@X-Ryl669 I need to think on your comment. These are all interesting/valuable comments in this thread.

For my exact use-case (and I don't think this is super rare), my read-only assets are... "mostly" read-only. Basically, under all normal operating conditions, the data is mmap'd and read-only. However, the assets may infrequently be updated over a slow-data-transfer-medium. In other projects, this could be updating some graphics as part of an OS-update or something.
I certainly understand that this introduces complexities and may not be feasible. If so, I'll pursue the "put contiguous data in another partition and have a littlefs file describe it" idea.

On my system, there's a MMU unit, so it's possible to map the flash address to the µc unit's address space.

Out of curiosity what µc are you using? Some devices I've seen have only ~1, 2, 3 XIP regions, which places a significant constraint on how you store things.

In a B-Tree like structure, I think it's possible to have multiple type of nodes, for different "block" sizes, without too much impact on the logic of the code (this requires tests for the node type where size is computed)

You're right this is easy on the B-Tree side, but it creates problems on the alloc side. If you alloc different sizes you risk fragmentation, where a small block sits in a large block and prevents it from being used in larger block allocations.
You can mitigate this with defragmentation (lfs_fs_defrag?), which moves small pieces of data together, but this adds more janitorial work to the system.

So, it might be possible to keep the new B-Tree structure to deal with some "super" 64kB blocks that can't be split anymore, yet, being able to move them

An annoying issue with fragmentation is that it can force larger allocations to fail, even if there is in theory enough storage available.
I guess you could make allocation failure trigger defragmentation, though this would quite severely increase the potential runtime spike of write operations.

First, the need for contiguous space is mainly required for reading. We could live without contiguous write support, IMHO.

But how do you create the contiguous file? Something somewhere needs to write? I suppose this is mostly an API question and not too important.

Additionally, a "memory mapped" bit could be added to the file or B-Tree node, so when this bit is set, no move of these blocks is allowed.

I was reaching the same conclusion, we really only need a "pin" bit on files to indicate that their data shouldn't be moved. This would allow for both contiguous files and more flexible MMU/virtual-mapped files. Though I don't know if mixed pinned/unpinned B-tree nodes is really useful.

They only need to be in RAM (not flash), since after reset, the mapping is gone.

Ah, this would be nice. But I think you need this bit to persist on flash so that contiguous files don't get broken up after a remount. Though this may not be a problem for systems with granular virtual mapping.

However, the assets may infrequently be updated over a slow-data-transfer-medium. In other projects, this could be updating some graphics as part of an OS-update or something.

I think this is very common. I guess whether or not to completely rewrite the filesystem in this situation is a bit of a user decision.
It would be nice to not need to reformat the filesystem, but with contiguous files this creates increasing fragmentation. Maybe lfs_fs_defrag is unavoidable conclusion...

I certainly understand that this introduces complexities and may not be feasible. If so, I'll pursue the "put contiguous data in another partition and have a littlefs file describe it" idea.

To be honest I think this features is going to be a minimum >=~1/2 year out in terms of timescales.

For my exact use-case (and I don't think this is super rare), my read-only assets are... "mostly" read-only. Basically, under all normal operating conditions, the data is mmap'd and read-only. However, the assets may infrequently be updated over a slow-data-transfer-medium. In other projects, this could be updating some graphics as part of an OS-update or something.

Yeah, same for our use case. Hence, this PR is designed to simply allow reserving a large contiguous block in one shot and giving direct access to the storage address. I expect most updates are infrequent and will not change size of assets either. Fragmentation may likely happen if there's too many assets, but it's not a killing issue. We can still fall back to regular fragmented files and load them into memory as regular assets instead of directly accessing from flash if that ever occurs.

I'm not sure fragmentation is induced here.
In order to simplify the work, maybe "just" having the ability to create a Littlefs partition with a super block size (that is, logical blocks that are an (fixed) concatenation of physical block) would work. In that case, no special modifications are required, only the ability to pin & unpin a block (to prevent erasing or moving it) and a "give the physical address of the list of logical block for this file" function
If that was possible, I think we could just have 2 partitions: one with logical block size = physical block size for regular files and the other with large logical blocks. For large assets, then the second partition would be used, and mmap'ed in a linear address space  via the MMU. Even if there isn't a MMU on the µc, I think it's completely possible to change the code using the asset to work in large page via a "virtual" address array mapping.
I'm using a ESP32 mainly, but also ARM's Cortex A chip that all supports MMU (albeit the former is rather dumb).

Fragmentation may likely happen if there's too many assets, but it's not a killing issue. We can still fall back to regular fragmented files and load them into memory as regular assets instead of directly accessing from flash if that ever occurs.

This is an interesting point, and clever. But I don't think it works in the XIP use case since you usually generally need the binary to be contiguous for the system to function.
And it's not something you can just throw storage at to solve. You would need enough storage for every file in the filesystem to take up the same size as the contiguous file before you can guarantee that the contiguous allocation won't fail.

The "pin" bit isn't there to tell if a file is contiguous or not. I was more thinking about maintaining a "pinned" open's file table in RAM.

A purely in-RAM flag is a nice idea, and would minimize on-disk changes. But for contiguous files I think you still need to persist this info on-disk. It's not an issue for the MMU use-case, but for contiguous files I assume there would be some application logic that expects certain files to be contiguous (boot bios -> mount fs -> xip "/image.bin").
If some work is done before the contiguous file is opened and pinned, say lfs_fs_gc() is called and that triggers wear-leveling and moves parts of "/image.bin" around, then the system could stop working.
The nice thing about the in-RAM pin is it can always be added as an optional feature without on-disk changes, but it only solves one of the use-cases presented here.

I'm using a ESP32 mainly, but also ARM's Cortex A chip that all supports MMU (albeit the former is rather dumb).

The ESP32 is a common target for littlefs. And they even get "dumber", heck, support for 8-bit MCUs is on the roadmap at some point : )

In order to simplify the work, maybe "just" having the ability to create a Littlefs partition with a super block size (that is, logical blocks that are an (fixed) concatenation of physical block that can't be split) would work. In that case, no special modifications are required, only the ability to pin & unpin a block (to prevent erasing or moving it) and a "give the physical address of the list of logical block for this file" function
If that was possible, I think we could just have 2 partitions: one with logical block size = physical block size for regular files and the other with large logical blocks. For large assets, then the second partition would be used, and mmap'ed in a linear address space via the MMU. Even if there isn't a MMU on the µc, I think it's completely possible to change the code using the asset to work in large page via a "virtual" address array mapping.

If I'm understanding this proposal correctly, can it mostly already be done? With the exception of pinning and getting the underlying block address?

littlefs already supports any block_size a multiple of the physical block_size, you should just need to change the configuration.

You can have two littlefs instances, one on the small block pool and one on the big block pool.

The downside is you need to choose what size to make the pools, but AFAICT this is unavoidable without introducing fragmentation in some way.

Yeah, same for our use case. Hence, this PR is designed to simply allow reserving a large contiguous block in one shot and giving direct access to the storage address. I expect most updates are infrequent and will not change size of assets either.

I wonder if the answer for XIP is to include optional on-demand defrag during contiguous allocations. This would get expensive ($O(FS \cdot n)$), but if these files are infrequently updated, that might be ok.
Humorously, if contiguous files don't participate in error-correction or wear-leveling, I think contiguous allocations end up functionally isomorphic to a hypothetical lfs_fs_shrink().

Humorously, if contiguous files don't participate in error-correction or wear-leveling, I think contiguous allocations end up functionally isomorphic to a hypothetical lfs_fs_shrink().

I think you're correct. I'm a bit loss in this conversation, but how I imagine it would go:

When a file is created/written, it has the option of being a contiguous mmap'd file or not. Under this circumstance, it doesn't participate in error-correction or wear-leveling.
Having several contiguous files throughout the littlefs partition could lead to fragmentation that prevents a file fitting, despite their being supposedly enough space for it do so.  To solve this, a defragmentation function would be necessary (and would rarely ever be called). This would make all the contiguous files be sequential at the beginning of the partition, with all the "free-flowing" normal blocks immediately after with no free-blocks intermingled.
Since all the data is now at the beginning of the littlefs partition without any gaps, the filesystem could be safely shrank. So this is a "two-birds-one-stone" feature.

If I'm understanding this proposal correctly, can it mostly already be done?

Yes. And you're completely right for your point about contiguous block and fragmentation issues.
If I understand correctly, there are numerous step that can be made toward the goal of a contiguous area in flash:
Step 1: (With physical random order for contiguous files)
Deal at app level, only possible for µc with MMU. Create 2 littlefs partitions, as described above.
This requires a API in littlefs to pin/lock a block list and get there address. The management of the contiguous area is then done by the application, virtually via the MMU. No need to have a permanent pin stored in flash.
Pros
If I understand correctly, it might also be the fastest option, since if the "logical" block size is the size of an erase block (or a multiple of so), you can skip erasing each page sequentially.
Cons
Requires 2 filesystem overhead. Won't work with non MMU system (or requires some change in the app code to deal with a manual virtual mapping)
Step 2: (With physical random order for contiguous files)
Move this logic inside a single littlefs partition. littlefs deals with 2 area in the partition, once that's made for small blocks and another that's dealing with large blocks only. There is no possible data exchange between those area, so no fragmentation can occur.
This would somehow require a way for the app to specify what kind of file to open (maybe using a O_HUGEFILE or something to refer to the latter area ?)
Pros
Same as above, should reduce the filesystem overhead to only a single partition
Cons
Requires change the on-disk format. Still require a MMU or a modification of the app code to deal with random large block.
Step 3: (Path toward contiguous file on the flash)
Start implementing a defragmentation process. I think having 2 pools here is way easier to help defragmenting. In effect, the "small" pool doesn't actually require to be defrag, but any free space in it can be used to move the blocks from the "large" large pool to reorder and sort them.
If done correctly, the writing/updating process would be:

(Optional) Delete previous (large) file
Launch a defragmentation step. littlefs is unusable while this process is performed (easier)
Defragmentation reorder the files in the large pool so that they are contiguous and compact
Write the new (large) file in the large pool by first allocating its size and then mmap'ing it or sequentially filling it
If, for some reason, the large file can't fit in the free space (because of a bad block, or whatever), fail here

Pro:
This is a true contiguous area so it can be used on non-MMU capable µc (like Cortex M)
Con:
The number of modification is (very) limited and it's a long and painful process. Might be ok for rare system update through.

if this is a read-only image, could you not allocate blocks for the contiguous files you need from the end before creating the filesystem, and store "fake" files in the filesystem that describe where they live on disk?

It's nowhere near a working state, but I began writing some python code implementing this idea.
https://github.com/BrianPugh/gnwmanager/blob/rom-fs/gnwmanager/romfs.py


remove "Future" chapter because it was outdated

Resolves #2294 Outdated Media Future

To me this change looks good - but it leads to another question:  what happened to the proposed future changes?

Are they already done? I could not find them in release notes.
Still planned for the future? If so, can we have links to them here?
Or cancelled?

Is this related to #85 and https://doi.org/10.3384/ecp15118653 ??

I think most of the described features are already available in MSL (Hubertus should confirm)

Clearly ok from my side to merge this, this is simply an outdated and forgotten piece of documentation. Re Martin: No, this capability is not yet in Modelica.Media, we never had proper multi-component multi-phase support. But changes would require a major new release of MSL since I see no way to make this fully backward compatible.


Prevent focus start point of being applied on server side which causes rendering error

In some scenarios, the onCleanup hook from the focus start point utility was called on server side, which leads to an error due to HTMLElement not defined (see #22). This moves the hook to a branch that should only be run on the client.

Had to close this now due to a massive rewrite, but thanks for contributing!


Added support to Poland mobile number pl-PL

I added support to Poland's mobile numbers, and I updated the tests and the README.md.

Coverage remained the same at 99.091% when pulling 4ffddfc2af20cfcd0c347e813fb51afcd1bd26c8 on FelipeNBrito:master into 73e5511e11c9fe08cba164149b5e25e694de9947 on chriso:master.

Thanks.


Log output from serving models for easier debugging

llama-cpp-python uses uvicorn and it turns out there's a different way to start the running models that uses uvicorn directly, and that makes it possible to pass logging configuration to uvicorn. Unfortunately it seems that logs that come directly from llama_cpp get thrown to stderr and it's not configurable
(https://github.com/abetlen/llama-cpp-python/blob/ae71ad1a147b10c2c3ba99eb086521cddcc4fad4/llama_cpp/_logger.py#L30)
This change will make it so that the llm tool is installed in the workstations impage with a default logging configuration file which writes logs to /var/log/localllm.log. I don't love how different the story is if you actually run the tool directly (you have to go out of your way to get the logging) but this seems okay for now at least.
The content of the logging config is from
https://gist.github.com/liviaerxin/d320e33cbcddcc5df76dd92948e5be3b
Fixes #16

thanks for the review @jerop !! i made the changes you suggested and then... FORGOT TO PUSH THEM so i'll add them into #19 instead


from nmt import nmt

Traceback (most recent call last):
File "F:/AI/RedditChatBot/nmt-chatbot-master/train.py", line 8, in
from nmt import nmt
File "F:\AI\RedditChatBot\nmt-chatbot-master\nmt\nmt.py", line 28, in
from . import inference
File "F:\AI\RedditChatBot\nmt-chatbot-master\nmt\inference.py", line 25, in
from . import gnmt_model
File "F:\AI\RedditChatBot\nmt-chatbot-master\nmt\gnmt_model.py", line 261, in
class GNMTAttentionMultiCell(tf.nn.rnn_cell.MultiRNNCell):
AttributeError: module 'tensorflow_core._api.v2.nn' has no attribute 'rnn_cell'
i have python 3.5.0
i have downloaded the new nmt file and replaced it with old one
and still have this problem. Please help

Please try installing other version of tensorflow, this framework is not written for tensorflow v2.0.

Is there any plan to update this framework for the latest version of tensorflow?
If not what is the preferred framework?


Update node LTS documentation

Category

[x] Content fix
[ ] New article
Related issues: mentioned in #2863

What's in this Pull Request?
Update regarding the NodeJS LTS information
Guidance
As of 2018-10-30 there is two LTS versions of NodeJS, added clearifying information that only 8.x is supported for SharePoint framework development.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/spfx/set-up-your-development-environment.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

100% valid fix, but unfortunately already merged similar update with other PR, so can't merge this. Thanks for your input.


issues with comp_g, intra_comp

Summing of masks
https://github.com/boris-kz/CogAlg/blob/e1d4c44ae364f86b24c2056a80c41d10dc2f2aba/frame_2D_alg/intra_comp.py#L202-L206
Please excuse me, but does this > 1 intended, or do you mean > 0? Even if it's intended, it has no effect on the results, because masks are summed when arrays are broadcast together anyway, like this:

Here's what I mean:

Could the summing of masks be put at the end? Could gg's mask be assigned to all other 2D arrays in the returned dert array?
Incorrect trigonometric identity
https://github.com/boris-kz/CogAlg/blob/e1d4c44ae364f86b24c2056a80c41d10dc2f2aba/frame_2D_alg/intra_comp.py#L217-L218
The cosine of the difference of two angles is:

(Picture taken from: https://en.wikipedia.org/wiki/List_of_trigonometric_identities)
So those two lines should be:
cos_da0__ = (cos2__ * cos0__) + (sin2__ * sin0__)  # top left to bottom right
cos_da1__ = (cos3__ * cos1__) + (sin3__ * sin1__) # top right to bottom left
No need to copy arrays
https://github.com/boris-kz/CogAlg/blob/e1d4c44ae364f86b24c2056a80c41d10dc2f2aba/frame_2D_alg/intra_comp.py#L232-L234
Copying will cause unnecessary overhead in this case. These arrays will be copied when called in ma.stack.

Summing of masks
https://github.com/boris-kz/CogAlg/blob/e1d4c44ae364f86b24c2056a80c41d10dc2f2aba/frame_2D_alg/intra_comp.py#L202-L206
Please excuse me, but does this > 1 intended, or do you mean > 0? Even if it's intended, it has no effect on the results, because masks are summed when arrays are broadcast together anyway, like this:

Here's what I mean:

Could the summing of masks be put at the end? Could gg's mask be assigned to all other 2D arrays in the returned dert array?

as i remembered, the value of 1 control the amount of unmasked value, if we increase it to 2 or 3, there would be lesser unmasked value. This is because when we sum them, their value might get more than 1

Correct me if I'm wrong, that line has no effect, given the operations 'til it reach gg__, dgy__, dgx__, mg__:
https://github.com/boris-kz/CogAlg/blob/e1d4c44ae364f86b24c2056a80c41d10dc2f2aba/frame_2D_alg/intra_comp.py#L208-L230
Even if you remove that line, results will be the same.

"The cosine of the difference of two angles is:
image
(Picture taken from: https://en.wikipedia.org/wiki/List_of_trigonometric_identities)
So those two lines should be:
cos_da0__ = (cos2__ * cos0__) + (sin2__ * sin0__) # top left to bottom right
cos_da1__ = (cos3__ * cos1__) + (sin3__ * sin1__) # top right to bottom left
"
Thanks Khanh!

@khanh93vn , what about cos_da from lines 136-145, is this correct:
    8-tuple of differences between center dert angle and rim dert angle:
    '''
    cos_da = np.stack((
              ((a__center[0] * a__center[1]) + (a__topleft[0] * a__topleft[1])),
              ((a__center[0] * a__center[1]) + (a__top[0] *  a__top[1])),
              ((a__center[0] * a__center[1]) + (a__topright[0] * a__topright[1])),
              ((a__center[0] * a__center[1]) + (a__right[0] * a__right[1])),
              ((a__center[0] * a__center[1]) + (a__bottomright[0] * a__bottomright[1])),
              ((a__center[0] * a__center[1]) + (a__bottom[0] * a__bottom[1])),
              ((a__center[0] * a__center[1]) + (a__bottomleft[0] * a__bottomleft[1])),
              ((a__center[0] * a__center[1]) + (a__left[0] * a__left[1]))
            ))

Correct me if I'm wrong, that line has no effect, given the operations 'til it reach gg__, dgy__, dgx__, mg__:
https://github.com/boris-kz/CogAlg/blob/e1d4c44ae364f86b24c2056a80c41d10dc2f2aba/frame_2D_alg/intra_comp.py#L208-L230
Even if you remove that line, results will be the same.

you mean which line?

you mean which line?

nevermind

@khanh93vn , what about cos_da from lines 136-145, is this correct:
    8-tuple of differences between center dert angle and rim dert angle:
    '''
    cos_da = np.stack((
              ((a__center[0] * a__center[1]) + (a__topleft[0] * a__topleft[1])),
              ((a__center[0] * a__center[1]) + (a__top[0] *  a__top[1])),
              ((a__center[0] * a__center[1]) + (a__topright[0] * a__topright[1])),
              ((a__center[0] * a__center[1]) + (a__right[0] * a__right[1])),
              ((a__center[0] * a__center[1]) + (a__bottomright[0] * a__bottomright[1])),
              ((a__center[0] * a__center[1]) + (a__bottom[0] * a__bottom[1])),
              ((a__center[0] * a__center[1]) + (a__bottomleft[0] * a__bottomleft[1])),
              ((a__center[0] * a__center[1]) + (a__left[0] * a__left[1]))
            ))

Apologise, I haven't check comp_r yet, I'll do it after having patched comp_g up.
Anyway, if there's no objections, I'll close this issue and push a patch for these 3 problems.