proposal: structs.NoCopy

Background: In general, copying of mutable Go data structures is a risky business. Go has no analogue of C++'s DISALLOW_COPY_CONSTRUCTORS macro or Rust's Copy trait, so it is always possible to copy values of any type. However, shallow copying may hide very subtle aliasing bugs:

The original and the copy can no longer both be used. For example, after copying a bytes.Buffer, subsequent mutations to either the original or the copy may cause unpredictable effects on the other one.
"Linear copying", in which the old value is no longer used after the copy, is typically less hazardous, but may still be unsafe or unpredictable. In the past, bytes.Buffer contained a small inline array that was used as the initial space for slice allocation. Copying a Buffer would cause the new copy to refer to the array field of the original. (This wasn't incorrect, but it was surprising and unintended.)
More complex data structures are not safe to use even after linear copying. For example, sync.WaitGroup and sync.Mutex contain a semaphore, which has special treatment in the runtime; a copy won't do. And strings.Builder uses unsafe tricks to avoid allocation; mutating a copy leads to crashes.

In general, Go programmers should assume, in the absence of explicit documentation or insider knowledge, that copying any arbitrary nonzero value of type T, where methods are defined on *T, is not safe.
However, for certain types (such as those mentioned above) the potential for latent serious bugs is high, and it is worth making an effort to detect copying mistakes statically (using cmd/vet's copylock check) and perhaps even dynamically, using the self-pointer technique demonstrated by strings.Builder.
Proposal: We propose to publish two new types, shown below, in the structs package. The first is merely an annotation for static analysis tools; the second additionally performs a dynamic check (at the cost of one extra word in your data structure).
We also propose to rename vet's copylock check to nocopy, and change it to look for structs.NoCopy fields instead of the presence of Lock/Unlock methods (which are an indirect proxy for non-copyability that dates from when the check applied only to Mutex and WaitGroup).
package structs

// NoCopy helps statically detect copying of non-zero values of
// mutable data types that must not be copied. (Examples include
// [strings.Builder] and [sync.WaitGroup].)
//
// Embed this type within your data structure to indicate that copying
// non-zero values of that type is not safe. This type has no dynamic
// effect, but static analysis tools may report an error when they
// encounter statements that copy a NoCopy value.
//
// See [NoCopyCheck] for a variant that additionally incorporates a
// run-time check.
//
// As a rule, unless explicitly documented to the contrary, you should
// assume that it is not safe to make copies of any non-zero value of
// a type T that defines methods on *T. Calling such a method may
// cause the data structure to incorporate its original address in
// some way--for example, by causing one field to point to
// another--such that copying the value violates invariants of the
// representation.
type NoCopy struct{}

// NoCopyCheck helps detect copying of mutable data types that must
// not be copied (for example, [strings.Builder]), using both runtime
// assertions and static checks.
//
// Embed a NoCopyCheck within your data structure, and call its check
// method at the start of each method:
//
//	type Buffer struct {
//		data   []byte
//		nocopy NoCopyCheck
//	}
//
//	func (b *Buffer) Write(data []byte) {
//		b.nocopy.Check()
//		b.data = append(b.data, data)
//	}
//
// If the structure has been copied, it will panic:
//
//	var old Buffer
//	old.Write([]byte("hello"))
//	new := old // Buffer values must not be copied!
//	old.Write([]byte("world"))
type NoCopyCheck struct {
	_ NoCopy
	ptr *NoCopyCheck // points to self after first call of Check
}

// Check panics if the NoCopyCheck has been copied since the previous
// call to Check, if any.
func (nocopy *NoCopyCheck) Check() {
	if nocopy.ptr == nil {
		nocopy.ptr = nocopy
	} else if nocopy.ptr != nocopy {
		panic("a struct with a NoCopyCheck field has been copied")
	}
}

Related:

https://github.com/golang/go/issues/23764
https://github.com/golang/go/issues/25907
https://github.com/golang/go/issues/47276
https://github.com/golang/go/issues/67265 (this proposal was split out from that one)

(Thanks to @ianthehat for pointing out that NoCopyCheck can contain a NoCopy so that vet need only consider the latter.)

Related Issues

runtime: add NoCopy documentation struct type? #8005 (closed)
proposal: Go 2: non-copyable structs by default #30450 (closed)
proposal: Go 2: NoCopy: official support in compiler (not just vet linter) #23764 (closed)
proposal: embed "noCopy" for bytes.Buffer and strings.Builder #25907 (closed)

Related Code Changes

std: pass bytes.Buffer and strings.Builder by pointer
bytes: document that Buffer values must not be copied

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

This subtle behavior change might be worth noting (assuming this API lands and strings.Builder is replaced to use structs.NoCopyCheck):
var s strings.Builder
s2 := s // nocopy vet warning
s2.WriteString("test") // copy runtime panic

Copying the zero value is safe, s2 := s, after this change s2 := s would cause a vet error.
Not sure whether it matters, leaving this here.

Exactly: the words "non-zero" are crucial.

Exactly: the words "non-zero" are crucial.

Yep, the current behaviour of the vet check does not seem to care whether it is zero value, hence the confusion.

Non-zero is a dynamic property. How is the static checker going to determine non-zero-ness?

Non-zero is a dynamic property. How is the static checker going to determine non-zero-ness?

The wording says "static analysis tools may report an error when they encounter statements that copy a NoCopy value"; the static check needn't insist that the value be non-zero.
But it could use a conservative heuristic. For example: a reference to a variable is assumed to be zero if and only if its declaration is local to the same function, and no control path from the def to the use refers to the variable as an l-value. But perhaps this is too conservative.

If this is about tools like vet catching things, you could just propose a magic name to put in a zero-sized field of the struct.
type T struct {
   something typed
   _do_not_copy_ struct{}
}

Such a convention will work fine for most of the need here. No need to add anything to the library. Your proposal doesn't carry its weight; it's too easy to say "it prevents bugs" to justify any complication.
I am saddened by all extra knobs, whistles, doodads, and festoonery that the library is accumulating these days. As Bjarne Stroustrup once said, looking the other direction, "If you want C++  you know where to get it."

While adding something to the standard library that has no effect other than being a hook for separate analysis tools is pretty weird, I think an advantage it has over just a naming convention is that a library symbol has an obvious place to go to find its documentation, and that documentation can then explain what it's for.
There are various little tricks and hacks that have become informal Go idiom over time, such as using an unnamed global variable to assert that type implements a particular interface so that the compiler will check it, and my experience has been that newcomers tend to come and ask me to explain what they mean because there is no clear path to their documentation. I would like there to be fewer of these things over time.
I think it could be justified to say that this isn't valuable enough to do at all and that instead the linter should just learn about specific standard library types like the ones mentioned in the proposal text, but if we agree that this is a problem worth solving then I'd personally rather solve it with a real language or library feature than to add another ad-hoc pattern that folks need to learn about by asking their peers.

Because it's not API at all and involves no types. You may have different criteria.

If this is so, what information does structs.NoCopy convey? Not just "assume this unsafe to copy" but "this is definitely unsafe to copy and we're going to vet check it." If the default is that things are unsafe, shouldn't there be some way to signal "this is okay to copy"?

You raise a good question. Based on the examples above, it would appear to signal "this type is particularly prone to bad things happening if you copy it, perhaps due to extralinguistic mechanisms (e.g. unsafe, scheduler)". But you're right that there's something odd about affirming what should be the default assumption, instead of flagging the non-default one.
Russ may be right that URLs are copyable, but that's not stated in its documentation, and URL has potential aliasing issues since at least one of its fields is indirect and it has public mutator methods like Parse. (Shallow copying followed by updating the copy's User field would cause unexpected effects on the original.) Existing practice has mostly been: read the source, and make a good guess based upon the name and API and doc comments. But URL's documentation should really state explicitly that it is copyable, if that is the intent, and ideally that intent should be recorded in a form amenable to tooling. But I can't imagine the inverse proposal (structs.Copyable) having much traction, too much code assumes that structs are copyable.

Is there some way we can we teach go vet what makes something safe or unsafe to copy and then look for that?

Any kind of indirection among the struct fields is a potential problem; it's hard to be more specific.


Add Scheduler tutorial

Scheduler is an integral part of an OS, can you make a tutorial for that?
Also make some small tutorials for raspberry pi, Os Wiki has rust bare metal tutorial, can you do something like that?
Regards

The generator/coroutine based method (untested) that I use in my kernel would make a great starting point for one.

The generator/coroutine based method (untested) that I use in my kernel would make a great starting point for a preemptive multitasking tutorial.

The last post of blog os is about async/await: https://os.phil-opp.com/async-await/

Would need to completely rewrite the blog from scratch in the RPi case however given how different ARM is from x86 at the assembly level.

True, there are quite some differences between ARM and x86 in terms of how many low level things that OSes require are implemented. For example page tables have a different format, the interrupt controller is different (and even differs between different soc's for arm although somewhat recently arm made a standard interrupt controller. not everyone uses it yet.), there is no vga text mode in arm (instead you have to write a display controller driver for the specific display controller of the soc to set up a framebuffer, or a serial port driver), there is no BIOS emulation of a PS/2 keyboard with dedicated interrupts (instead you likely have to write a full USB driver, or a serial port driver) and much more. For display output and keyboard input you may be able to use UEFI instead, but not every arm system has the option to run UEFI and in the case of keyboard input you must not exit boot services, which means that you can't do a lot of things expected from an OS like running usermode programs. Before exitting boot services you are effectively a program running under the UEFI "OS".

The generator/coroutine based method (untested) that I use in my kernel would make a great starting point for a preemptive multitasking tutorial.

The last post of blog os is about async/await: https://os.phil-opp.com/async-await/

That's cooperative. This unstable feature allows one to use combination of an enum and yield () within the coroutine body to block execution from the outside.


[python] Use bindings for SparseNDArray readpath

Issue and/or context:
https://github.com/single-cell-data/TileDB-SOMA/issues/2077
Changes:
Use clib.SparseNDArray for read path, replacing usage of tiledb.Array.
These changes were picked out of https://github.com/single-cell-data/TileDB-SOMA/pull/1817.
Notes for Reviewer:
These changes are on top of https://github.com/single-cell-data/TileDB-SOMA/pull/2132.

Needs to be updated to use SOMAContext once #2158 is merged.

This is ready for review.

[sc-40220]


[Pre-Release:] Remove warning suppressions from Azure.AI.FormRecognizer.csproj

Seeing we have warning in various places, and removing them involves design work, I will say we should estimate at least 1 day for this

Updated.


update go mod to 1.20

only change here is updating the go mod to 1.20. 1.20 does not support go mod tidy command so I am going to close this for now. If security scans pick up something specific for cert manager operator then I will revisit


Update to Typescript 2.0

Update the package to include TS2 as well as the gulp plugin that builds the project. As part of this we should also update the tsconfig file to use the new Glob support.

See 292cb6eba8e858a5edd84a613194939f319d1f25


Expose firmware path which can be different based on platform - ie ubuntu

Fixes #2

Thank you for your contribution @robcxyz! You are right, I was biased by working on Fedora/CentOS while creating it, thanks again for pointing this out!


fix: fix styles for Markdown and Monaco Editor, added light theme

fix theme styles for Markdown and Monaco Editor
added support for the light theme

Dark

Light

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Remove the implementationOnly dance

Motivation:
When we added implementationOnly imports a while back, we supported
Swift 5.0, and so we needed to guard their availability behind a
compiler check. We only support 5.2 and later now, so that check is
redundant and can go away.
Modifications:

Removed compiler checks around imports of our C libraries.

Result:
Cleaner code

Do we maybe want to test in our soundness script that CNIOBoringSSL is always imported @_implementationOnly?

We don't need to: the Swift compiler will warn if we're inconsistent, and that'll produce an error in CI.

We don't need to: the Swift compiler will warn if we're inconsistent, and that'll produce an error in CI.

Right! I forgot!


Ugly hotfix for iOS notch color

Technically addresses #519 but fails to consider background images and theme switch button.

@Capprin thanks for submitting this.

@Capprin I will not have time to work on this and I don't think this solution is good. Are you planning to fix my comments? Otherwise I suggest we close this PR and discuss further in the ticket before making code changes.

Closing this PR, found a way to do it automatically without requiring configs.
https://github.com/nunocoracao/blowfish/pull/546


[Bug]: PersistentBottomNavBarV2 Overlay Issue with Page Content

Version
5.2.3
Flutter Doctor Output
[✓] Flutter (Channel stable, 3.22.2, on macOS 14.5 23F79 darwin-x64, locale en-GB)
    • Flutter version 3.22.2 on channel stable at /Users/aungmyooo/Development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 761747bfc5 (4 weeks ago), 2024-06-05 22:15:13 +0200
    • Engine revision edd8546116
    • Dart version 3.4.3
    • DevTools version 2.34.3

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/aungmyooo/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.4)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15F31d
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] VS Code (version 1.91.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.92.0

[✓] Connected device (3 available)
    • Aung Myo’s iPhone (mobile) • 00008120-0006612E0198A01E • ios            • iOS 17.5.1 21F90
    • macOS (desktop)            • macos                     • darwin-x64     • macOS 14.5 23F79 darwin-x64
    • Chrome (web)               • chrome                    • web-javascript • Google Chrome 126.0.6478.127

[✓] Network resources
    • All expected network resources are available.

• No issues found!

What platforms are you seeing the problem on?
iOS
What happened?
Description:
After upgrading to the latest version of PersistentBottomNavBarV2, we've encountered an issue where the GestureDetector containing the version info widget is overlaid by the bottom navigation bar. This forces us to add a SizedBox with a height of 100 at the bottom of every page to prevent the content from being hidden.
Actual Behavior:
The version info widget is overlaid by the bottom navigation bar, requiring a manual workaround by adding a SizedBox with a height of 100 at the bottom of each page.
Temporary Workaround:
Adding a SizedBox with a height of 100 at the bottom of every page to prevent the content from being hidden by the PersistentBottomNavBarV2.
Steps to reproduce
Upgrade to the latest version of PersistentBottomNavBarV2.
Add a GestureDetector at the bottom of a page to display version information.
Observe that the version info widget is overlaid by the bottom navigation bar.
Expected Behavior:
The content at the bottom of the page, specifically the version info widget, should be fully visible and not overlaid by the PersistentBottomNavBarV2.
Code to reproduce the problem
Widget build(BuildContext context) {
    onChanged(bool darkTheme) {
      AppTheme selectedTheme =
          darkTheme == true ? AppTheme.dark : AppTheme.light;
      BlocProvider.of<ThemeBloc>(context)
          .add(ThemeEvent(appTheme: selectedTheme));
      Preferences.saveTheme(selectedTheme);
      setState(() {
        isDarkTheme = darkTheme;
      });
    }

    return Scaffold(
      backgroundColor: Theme.of(context).colorScheme.surface,
      appBar: CustomAppBar(
        title: kSetting,
        onBackPressed: () {
          Navigator.pop(context);
        },
      ),
      body: Container(
        padding: const EdgeInsets.all(10),
        child: Column(children: [
          Expanded(
            child: ListView(
              children: [
                Column(
                  children: [
                    const SizedBox(
                      height: 10,
                    ),
                    const Row(
                      children: [
                        Icon(Icons.mode_night_outlined),
                        SizedBox(
                          width: 10,
                        ),
                        Text(
                          kDarkMode,
                          style: TextStyle(
                              fontSize: 18, fontWeight: FontWeight.bold),
                        ),
                      ],
                    ),
                    Divider(
                      height: 20,
                      thickness: 1,
                      color: Theme.of(context).dividerColor,
                    ),
                    const SizedBox(
                      height: 5,
                    ),
                    Padding(
                      padding: const EdgeInsets.symmetric(
                        vertical: 0,
                        horizontal: 20,
                      ),
                      child: Row(
                        mainAxisAlignment: MainAxisAlignment.spaceBetween,
                        children: [
                          const Text(
                            kDark,
                            style: TextStyle(
                              // fontSize: 18,
                              fontWeight: FontWeight.w500,
                              // color: Colors.grey[600],
                            ),
                          ),
                          Transform.scale(
                            scale: 0.7,
                            child: CupertinoSwitch(
                              activeColor: Theme.of(context).primaryColor,
                              value: isDarkTheme,
                              onChanged: (bool newValue) {
                                onChanged(newValue);
                              },
                            ),
                          ),
                        ],
                      ),
                    ),
                    const SizedBox(
                      height: 50,
                    ),
                  ],
                ),
              ],
            ),
          ),
          GestureDetector(
            onTap: () {
              ScaffoldMessenger.of(Globals.navigatorState.currentState!.context)
              ..hideCurrentSnackBar()
              ..showSnackBar(SnackBar(
                content: Text(apiUrl!),
                duration: const Duration(seconds: 3),
              ));
            },
            child: Center(
              child: Container(
                padding:
                    const EdgeInsets.symmetric(horizontal: 20, vertical: 10),
                decoration: BoxDecoration(
                    borderRadius: BorderRadius.circular(20),
                    border: Border.all(color: Theme.of(context).primaryColor)),
                child: FutureBuilder<PackageInfo>(
                  future: PackageInfo.fromPlatform(),
                  builder: (context, snapshot) {
                    return Text(
                      snapshot.hasData
                          ? ('Version: ${snapshot.data!.version}.${snapshot.data!.buildNumber}')
                          : '',
                      //style: const TextStyle(color: Colors.black),
                    );
                  },
                ),
              ),
            ),
          ),
          const SizedBox(
            height: 100,
          )
        ]),
      ),
    );
  }
}

### Relevant log output

_No response_

### Screenshots

![IMG_2D3D45E941DD-1](https://github.com/jb3rndt/PersistentBottomNavBarV2/assets/152832891/b6d66386-2532-438b-8cf7-fac2ce8148d9)

navBarOverlap: const NavBarOverlap.custom(overlap: 0.0),
It is fixed.


Simplify RigidAlignmentModule and GroupWiseRegistrationModule

When built against Slicer, the latest version of ShapePopulationViewer (SPV) is built as a loadable module.
This means that models already loaded in Slicer can easily be loaded in SPV, models can also be drag-and-dropped and loaded automatically.
This means that that the workflow in RigidAlignmentModule (or GroupWiseRegistrationModule) should be  updated.
Currently the workflow is expected to be the following:

Display shape in shape population viewer by starting the CLI and expect the user to select entries from the CSV dialog. See here
Run rigid alignment CLI (or GROUPS CLI)
Run surf remesh CLI
Display shape in shape population viewer by starting the CLI and expecting the user to select entries from the CSV dialog. See here

A possible workflow could be the following:

Run rigid alignment CLI (or GROUPS CLI)
Run surf remesh CLI
Automatically select SPV and load shapes (without creating intermediate CSV files)

Thanks @jcfr
@styner @juanprietob @mmmosman88 Could you please let us know what is the behavior you wanted and why in this module so we can decide how / if needs modification before including in SALT?
Thank you!
Bea

Would also appreciate if you could review changes in https://github.com/NIRALUser/GROUPS/pull/25 and also provide test data for validating the workflow.

@bpaniagua The latter option is OK for me for the expected behavior
@jcfr the same dataset that is used for output of SPHARM can be used here.

This has all been addressed.


⚠ ClamAV has degraded performance

In 463ed15, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 267 ms

Resolved: ClamAV performance has improved in f006250 after 21 minutes.


Change structure of displayResults so that APPEND happens AFTER the loop and not it it.

.. As per @uberponky suggestion.. It should improve performance and is better practice!

Done


Fix the variables fetched from the data endpoint.

The data is returned as an array of objects. Identify the correct data variable for the repository URL and number of stars. Change the react code accordingly.

Hi @Lakhwinderr . Please assign this to me. I would like to work on this

HI @MuhammedJaseemVK,
Thanks for your interest in contributing! You can check out our CONTRIBUTING.md for guidelines on how to write a pull request (PR). Since this involves updating the React code, feel free to include screenshots or images to demonstrate your changes effectively.
If you have any questions, don’t hesitate to ask!
Looking forward to your contribution!


Not a bug but a help post

I am using this code to generate YAML file for my synthetic data base to learn how to use fides:
fidesctl generate dataset db fides_resources/fidesfaker_dataset.yml  --connection-string Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=fidesdatabase;Data Source=DESKTOP-ET8QFN2\SQLEXPRESS

This is the image.
The error I get is at the bottom
Could someone please help me? Thank you.

Version:
OS:
Python Version:
Docker Version:

Additional context
Traceback (most recent call last):
File "/fides/src/fidesctl/ctl/core/utils.py", line 59, in get_db_engine
engine = sqlalchemy.create_engine(connection_string)
File "", line 2, in create_engine
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned
return fn(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 530, in create_engine
u = _url.make_url(url)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/url.py", line 715, in make_url
return _parse_rfc1738_args(name_or_url)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/url.py", line 776, in _parse_rfc1738_args
raise exc.ArgumentError(
sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string 'Provider=SQLOLEDB.1'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/fidesctl", line 8, in
sys.exit(cli())
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/fides/src/fidesctl/cli/utils.py", line 192, in wrapper_func
raise err
File "/fides/src/fidesctl/cli/utils.py", line 188, in wrapper_func
return ctx.invoke(func, ctx, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/fides/src/fidesctl/cli/commands/generate.py", line 71, in generate_dataset_db
_dataset.generate_dataset_db(
File "/fides/src/fidesctl/ctl/core/dataset.py", line 302, in generate_dataset_db
db_datasets = generate_db_datasets(connection_string=connection_string)
File "/fides/src/fidesctl/ctl/core/dataset.py", line 273, in generate_db_datasets
db_engine = get_db_engine(connection_string)
File "/fides/src/fidesctl/ctl/core/utils.py", line 61, in get_db_engine
raise Exception("Failed to create engine!") from err
Exception: Failed to create engine!
bash: Integrated: command not found
bash: Persist: command not found
bash: Initial: command not found
bash: Data: command not found

Also used this:
fidesctl generate dataset db fides_resources/fidesfaker_dataset.yml  --connection-string Server=C:\Program Files\Microsoft SQL Server\MSSQL15.SQLEXPRESS\MSSQL;Database=fidesdatabase;Uid=Administrator;Pwd=******;
the error I get with this is:
Usage: fidesctl generate dataset db [OPTIONS] OUTPUT_FILENAME
Try 'fidesctl generate dataset db -h' for help.
Error: Got unexpected extra arguments (FilesMicrosoft SQL ServerMSSQL15.SQLEXPRESSMSSQL)

@ThomasLaPiana
Quotations didnt work, it gave the same exception error as before.
Checked my error: Could not parse rfc1738 URL from string
My problem is that not the quotations but the connection string is wrong which I do not understand why.

This is my current database. I do not think something is wrong with it but it would be very nice to comment a document here or tell me how can I extract connection string from this.

@LawyerDev68 generally the connection string should look something like found in this post
your current connection string is not valid as a URL which is where the error is coming from

@ThomasLaPiana I installed postgresql to follow up fides tutorial properly.

my postgresql database
--connection-string postgresql://postgres:[password]@localhost:5432/fidesdatabase
I checked the documentation here: https://hasura.io/learn/database/postgresql/installation/postgresql-connection-string/
and followed the exact steps
I am still getting the same error. Is there something I didnt see it properly?

I'm running the following command and seeing it work:
root@64cc06461b53:/fides# fides scan dataset db --connection-string postgresql+psycopg2://postgres:fides@fides-db:5432/fides_test
Loading config from: /fides/.fides/fides.toml
Loading resource manifests from: .fides/
Taxonomy successfully created.
Loaded the following dataset manifests:
        fides_db
        fides_cache
Successfully scanned the following datasets:
        public

Annotation coverage: 100%

So my biggest guess here is that the password has something that isn't getting escaped properly 🤔
Do you get the same error when you quote it?

@ThomasLaPiana I had problems with password about this database, but then again if I'm able to enter the database with the password I typed through postgresql I think I should be able to do that in code too. But yeah I'll check couple of passwords, no password at all etc. Also yeah I tried to put it in the middle of the quotes but I'll try again with the new stuff too. Cant do that rn but will report here what happened

This looks like a password issue but I couldnt wrap my head around it tbh

@LawyerDev68 can you run the following command to verify postgres is up and ready at the expected host and port: pg_isready --host=localhost --port=5432? You should see a response of localhost:5432 - accepting connections

This is an issue with the command. If it was an issue with connecting to the database the message would be localhost:5432 - no response. I think the path is C:\Program Files\PostgreSQL\14\bin. @ThomasLaPiana do you know if this is correct on Windows? You will also probably have to close and reopen your shell to pick up the new path.

@sanders41 tried to change the path to the way you proposed and restarted the shell, didnt work.

psql variable

@sanders41 @ThomasLaPiana Also this is important to say, I installed postgresql plugin to my vs code and found my database through the plugin. So I think that means my connection string does not have any issue. It might be a bug.

Ok lets try one more thing. If you run netstat -a -o do you see <IP_ADDRESS>:5432 in the Local Address list?

bash: netstat: command not found
@sanders41  Getting this error, but VS code is in the path, I can use shell to install other stuff like Faker etc.

bash? You are on windows right? You should be able to run that command from powershell.

@sanders41 Yeah I'm on windows and I tried it on powershell but it didnt work. Hmm somethings missing in my system, what should I do? Any recommendations?

@LawyerDev68 I tested locally with a postgres database at localhost in case that was the issue, but it completed successfully:
fides> fides scan dataset db --connection-string postgresql+psycopg2://postgres:fides@localhost:5432/fides_test
Loading config from: .fides/fides.toml
Loading resource manifests from: .fides/
Taxonomy successfully created.
Loaded the following dataset manifests:
        fides_db
        fides_cache
Successfully scanned the following datasets:
        public

Annotation coverage: 100%
fides>

for us to better help you debug, can you show me the entire, unredacted command you're running for scanning now?

Alright this is the entire code.

I have connected the database through postgresql extension in vs without a problem with the same parameters I entered below with the password etc.
PS C:\Users\Administrator\Desktop\Ders ve İş\VS> cd fides/
PS C:\Users\Administrator\Desktop\Ders ve İş\VS\fides> nox -s dev
nox > Running session dev
nox > Creating virtual environment (virtualenv) using python.exe in .nox\dev
nox > docker build --target=dev --tag ethyca/fidesctl:local .
[+] Building 4.3s (23/23) FINISHED
=> [internal] load build definition from Dockerfile                                                                                0.0s
=> => transferring dockerfile: 32B                                                                                0.0s
=> [internal] load .dockerignore                                                                                0.1s
=> => transferring context: 35B                                                                                0.0s
=> [internal] load metadata for docker.io/library/python:3.9-slim-bullseye                                                                                2.0s
=> [base 1/2] FROM docker.io/library/python:3.9-slim-bullseye@sha256:be8c64a08749503880e77755a1089272d4a201e395142aabd4bab54e7f547376                                                                                0.0s
=> [internal] load build context                                                                                2.0s
=> => transferring context: 51.49MB                                                                                2.0s
=> CACHED [base 2/2] RUN pip install -U pip                                                                                0.0s
=> CACHED [builder  1/15] RUN :     && apt-get update     && apt-get install     -y --no-install-recommends     curl     git     vim     g++     gnupg     gcc     && apt-get clean     && rm -rf /var/lib/apt/lists/*                                         0.0s
=> CACHED [builder  2/15] RUN :     && apt-get update     && apt-get install     -y --no-install-recommends     libpq-dev     && apt-get clean     && rm -rf /var/lib/apt/lists/*                                                                              0.0s
=> CACHED [builder  3/15] RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -                                                                                0.0s
=> CACHED [builder  4/15] RUN curl https://packages.microsoft.com/config/debian/11/prod.list | tee /etc/apt/sources.list.d/msprod.list                                                                                0.0s
=> CACHED [builder  5/15] RUN :     && apt-get update     && apt-get install     -y --no-install-recommends     apt-transport-https     unixodbc-dev     mssql-tools     && apt-get clean     && rm -rf /var/lib/apt/lists/*                                   0.0s
=> CACHED [builder  6/15] COPY dev-requirements.txt dev-requirements.txt                                                                                0.0s
=> CACHED [builder  7/15] RUN pip install -r dev-requirements.txt                                                                                0.0s
=> CACHED [builder  8/15] COPY requirements.txt requirements.txt                                                                                0.0s
=> CACHED [builder  9/15] RUN pip install -r requirements.txt                                                                                0.0s
=> CACHED [builder 10/15] COPY optional-requirements.txt optional-requirements.txt                                                                                0.0s
=> CACHED [builder 11/15] RUN pip install -r optional-requirements.txt                                                                                0.0s
=> CACHED [builder 12/15] COPY . /fides                                                                                0.0s
=> CACHED [builder 13/15] WORKDIR /fides                                                                                0.0s
=> CACHED [builder 14/15] RUN git rm --cached -r .                                                                                0.0s
=> CACHED [builder 15/15] RUN git reset --hard                                                                                0.0s
=> CACHED [dev 1/1] RUN pip install --no-deps -e ".[all,mssql]"                                                                                0.0s
=> exporting to image                                                                                0.1s
=> => exporting layers                                                                                0.0s
=> => writing image sha256:1b5d4328105ee2c37e83c454badf76a95e80d4bef86a2f55c6f69ea05a2edf8e                                                                                0.1s
=> => naming to docker.io/ethyca/fidesctl:local                                                                                0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
nox > docker build --target=frontend --tag ethyca/fidesctl:local-ui .
[+] Building 2.1s (10/10) FINISHED
=> [internal] load build definition from Dockerfile                                                                                0.0s
=> => transferring dockerfile: 32B                                                                                0.0s
=> [internal] load .dockerignore                                                                                0.1s
=> => transferring context: 35B                                                                                0.0s
=> [internal] load metadata for docker.io/library/node:16                                                                                1.7s
=> [internal] load build context                                                                                0.1s
=> => transferring context: 28.09kB                                                                                0.0s
=> [frontend 1/5] FROM docker.io/library/node:16@sha256:b35e76ba744a975b9a5428b6c3cde1a1cf0be53b246e1e9a4874f87034222b5a                                                                                0.0s
=> CACHED [frontend 2/5] WORKDIR /fides/clients/ctl/admin-ui                                                                                0.0s
=> CACHED [frontend 3/5] COPY clients/ctl/admin-ui/ .                                                                                0.0s
=> CACHED [frontend 4/5] RUN npm install                                                                                0.0s
=> CACHED [frontend 5/5] RUN npm run export                                                                                0.0s
=> exporting to image                                                                                0.1s
=> => exporting layers                                                                                0.0s
=> => writing image sha256:a60ee6a6e094221ce4b10becd3ca4900365572af6173dfca8a9489a640bb3f2f                                                                                0.1s
=> => naming to docker.io/ethyca/fidesctl:local-ui                                                                                0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
nox > docker-compose up -d fidesctl-ui
time="2022-10-08T09:49:47+03:00" level=warning msg="The "FIDESCTL__CLI__ANALYTICS_ID" variable is not set. Defaulting to a blank string."
[+] Running 3/3

Container fides-fidesctl-db-1  Healthy                                                                                16.9s
Container fides-fidesctl-1     Started                                                                                18.2s
Container fides-fidesctl-ui-1  Started                                                                                19.6s
nox > docker-compose run --rm -e FIDESCTL__CLI__ANALYTICS_ID --user=root fidesctl /bin/bash
time="2022-10-08T09:50:07+03:00" level=warning msg="The "FIDESCTL__CLI__ANALYTICS_ID" variable is not set. Defaulting to a blank string."
[+] Running 1/0
Container fides-fidesctl-db-1  Running                                                                                0.0s
root@d06ab49436eb:/fides# fidesctl init

███████╗██╗██████╗ ███████╗███████╗ ██████╗████████╗██╗
██╔════╝██║██╔══██╗██╔════╝██╔════╝██╔════╝╚══██╔══╝██║
█████╗  ██║██║  ██║█████╗  ███████╗██║        ██║   ██║
██╔══╝  ██║██║  ██║██╔══╝  ╚════██║██║        ██║   ██║
██║     ██║██████╔╝███████╗███████║╚██████╗   ██║   ███████╗
╚═╝     ╚═╝╚═════╝ ╚══════╝╚══════╝ ╚═════╝   ╚═╝   ╚══════╝
Initializing Fidesctl...
Directory './.fides' already exists.
Configuration file already exists: ./.fides/fidesctl.toml
For example policies and help getting started, see:
https://ethyca.github.io/fides/guides/policies/
Fidesctl initialization complete.
root@d06ab49436eb:/fides# fides scan dataset db --connection-string postgresql://postgres:***********@localhost:5432/fidesdatabase
bash: !@localhost: event not found
root@d06ab49436eb:/fides# set +H
root@d06ab49436eb:/fides# fides scan dataset db --connection-string postgresql://postgres:**********@localhost:5432/fidesdatabase
Loading resource manifests from: .fides/
Taxonomy successfully created.
Loaded the following dataset manifests:
public
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3243, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 310, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 868, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 476, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 146, in do_get
self.dec_overflow()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit
compat.raise(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 256, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 371, in init
self.__connect()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 666, in connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit
compat.raise(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 590, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 584, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python3.9/site-packages/psycopg2/init.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (<IP_ADDRESS>) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/fides/src/fidesctl/ctl/core/utils.py", line 64, in get_db_engine
with engine.begin() as connection:
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3025, in begin
conn = self.connect(close_with_result=close_with_result)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3197, in connect
return self._connection_cls(self, close_with_result=close_with_result)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 96, in init
else engine.raw_connection()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3276, in raw_connection
return self._wrap_pool_connect(self.pool.connect, _connection)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3246, in _wrap_pool_connect
Connection.handle_dbapi_exception_noconnection(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2100, in handle_dbapi_exception_noconnection
util.raise(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3243, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 310, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 868, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 476, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 146, in do_get
self.dec_overflow()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit
compat.raise(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 256, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 371, in init
self.__connect()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 666, in connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit
compat.raise(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 590, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 584, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python3.9/site-packages/psycopg2/init.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused
Is the server running on host "localhost" (<IP_ADDRESS>) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
(Background on this error at: https://sqlalche.me/e/14/e3q8)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/fides", line 8, in
sys.exit(cli())
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/fides/src/fidesctl/cli/utils.py", line 192, in wrapper_func
raise err
File "/fides/src/fidesctl/cli/utils.py", line 188, in wrapper_func
return ctx.invoke(func, ctx, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(args, kwargs)
File "/fides/src/fidesctl/cli/commands/scan.py", line 75, in scan_dataset_db
_dataset.scan_dataset_db(
File "/fides/src/fidesctl/ctl/core/dataset.py", line 248, in scan_dataset_db
db_datasets = generate_db_datasets(connection_string=connection_string)
File "/fides/src/fidesctl/ctl/core/dataset.py", line 273, in generate_db_datasets
db_engine = get_db_engine(connection_string)
File "/fides/src/fidesctl/ctl/core/utils.py", line 67, in get_db_engine
raise Exception(f"Database connection failed with engine:\n{engine}!") from err
Exception: Database connection failed with engine:
Engine(postgresql://postgres:@localhost:5432/fidesdatabase)!
root@d06ab49436eb:/fides#

can you try directly connecting to it via a python shell with sqlalchemy?
Follow this: https://docs.sqlalchemy.org/en/14/core/engines.html#postgresql
and then try to use that engine to talk to the database, such as:
engine = sqlalchemy.create_engine(connection_string)
with engine.begin() as connection:
    connection.execute("SELECT 1")

I think it works @ThomasLaPiana

Didnt receive any error message in any terminals

Disclaimer: Writing this as a feedback and explaining my own experience.
I think I wont be trying to use Fides for a while. Dealing with docker, linux systems etc. were too complicated for me. I am sure this is a good thing for senior devs and quality of the systems but for me, entrance bar was a bit high. I had a friend (Bachelors at Electrical Engineering, Masters at Data Science, has his own company, also a dev etc.) Said that this is too complicated for people like me and even mid-tier devs.
Then again its an open-source system so I'm not even complaining. But yeah, just saying.

Disclaimer: Writing this as a feedback and explaining my own experience. I think I wont be trying to use Fides for a while. Dealing with docker, linux systems etc. were too complicated for me. I am sure this is a good thing for senior devs and quality of the systems but for me, entrance bar was a bit high. I had a friend (Bachelors at Electrical Engineering, Masters at Data Science, has his own company, also a dev etc.) Said that this is too complicated for people like me and even mid-tier devs. Then again its an open-source system so I'm not even complaining. But yeah, just saying.

We really appreciate the candid feedback, and are actively working to make the user experience better so that it doesn't have that feeling of impenetrability. We believe privacy is a human right, and therefore the tooling to make that a reality should be as accessible as possible 🙂 Thank you again for taking the time to try things out and even open an issue to describe your experience, it's really valuable for us!


Allow subscribing to a specific businessKey

Right now for when a worker is configured, it subscribes from all businessKeys for a list of topics.
I have a use case where there are multiple process instances running with different business keys.
This behavior is supported  in the fetchAndLock REST call.

Hello @bangeneticalgorithms
Can you please add the PR for this functionality?
Thank you.


Add compatibility with Python 3

Made some minor changes to make it work with Python 3.

Many thanks!


Add docs to web_endpoints and remove /openapi.json from default

Describe your changes
Linear ticket
Allows activation of Swagger/Redoc docs by passing a flag to web_endpoint. Also disables /openapi.json route by default. This is implemented internally via a new boolean field on WebhookConfig.
Tested by running against a devbox Modal instance.
Backward/forward compatibility checks
Don't think I fully understand the implications here, so looking for review!

[ ] Client+Server: this change is compatible with old servers
[ ] Client forward compatibility: this change ensures client can accept data intended for later versions of itself

Changelog

web_endpoints now have the option to include interactive SwaggerUI/redoc docs by setting docs=True
web_endpoints no longer include an OpenAPI JSON spec route by default

Does anything need to happen server-side here?

Does anything need to happen server-side here?

Not that I could tell! Didn't find any usage of WebhookConfig that needed to be changed. And I ran integration tests against the updated protos and did a spot check with the dev server running the basic_web examples with and without the docs flag set.

Yeah that sounds right! Do we need to change the protos at all then?

I think we do need to change the protobuf to add this flag, we just don't need to change any server-side logic.
If I grok the system, the explanation is: we need to communicate configuration information from the client to the server and then to the worker. Matching other patterns I found, in this implementation a user Python process provides an argument to a public decorator (@web_endpoint) that ends up communicated to a private method (webhook_asgi_app) running inside a worker. The worker consumes it via the container entrypoint. The serialization format for those messages is given by the protos.
Here's my understanding of the backward/forward compatibility concerns.
For old clients, everything is fine: the new field is added on the server (as False) but then dropped by the client on the worker.
If for some reason the worker client was a newer version than the user client, it would read the value False and continue to not show docs. The /openapi.json route would go away, but that was not documented behavior and I don't think it was intended.
For new clients talking to old servers, the field will be dropped by the server and then not present when the worker client reads it. That would lead to docs not being served even when requested. So I assume that when we update the protos, we first propagate that to all servers, then release the new client version? Or I missed something!


Adding no longer maintained note

Upload no longer maintained file/note.

@csmoore - could you also do this one?
Thanks!


Fix zap scan docker name

Trello card
https://trello.com/c/TzzGdpx3/1826-git-fix-zap-scan
Context
Build workflow zap scan fails as docker name has changed
Changes proposed in this pull request
Update workflow
Guidance to review
Confirm scan runs in build workflow
see https://github.com/DFE-Digital/get-into-teaching-app/actions/runs/9156401185/job/25171239319

 Git - fix zap scan


Wasmtime wasi-sockets implementation: UDP

@sunfishcode @pchickey should this be reassigned to me?
Since #7029 has landed, I believe this is ready to be worked on.


The cursor jumps to the end of the line when renaming a task.

My actions before raising this issue

[x] Read/searched the docs
[x] Searched past issues

Expected Behaviour
When renaming a task, the cursor does not jump to the end of the line.
Current Behaviour
When renaming a task, if put the cursor anywhere other than the end of the line and start typing characters, the cursor jumps to the end of the line.
Possible Solution

Steps to Reproduce (for bugs)

Create an annotation task.
Click to "Edit" to edit the task name.
Put the cursor anywhere in the line except at the end and enter any character

Context

Your Environment

Git hash commit (git log -1): 4edabdb3ed34b0902140493f78634770936047a9
Docker version docker version (e.g. Docker 17.0.05): 19.03.13
Are you using Docker Swarm or Kubernetes? No
Operating System and version (e.g. Linux, Windows, MacOS): Windows 10
Code example or link to GitHub repo or gist to reproduce problem:
Other diagnostic information / logs:
Logs from `cvat` container

Next steps
You may join our Gitter channel for community support.

@dvkruchinin, it's AntD bug. Reported to their repo https://github.com/ant-design/ant-design/issues/28487.

Has been fixed two days ago. Probably need to update antd in CVAT.


Feeds published in reverse order display in rev and and sometimes show only old feeds

Feed URL
https://www.smashingmagazine.com/feed
Add-on version
3.0
Describe the bug
Certain feeds (like Smashing Magazine) that are published in ASC order by the publisher display in reverse and/or show old posts in Livemarks.
I haven't worked with RSS much on the development side, so forgive my lack of knowledge. Basically I'm not sure if the ordering of a published feed is important and whether it's possible for the feed reader to handle feeds published in reverse.
A problem I've seen a few times with Livemarks is when a feed is published in ascending order, it doesn't seem to care about the publish date and just displays in the order received (the older ones at the top, and cuts of the rest if more are published than Live Mark's limit... which I believe 25). With SM for example, that means I always see only the oldest 25 of the 50+ that they publish.
To Reproduce

subscribe to the feed on smashingmagazine (above link).
go to the home page of https://smashingmagazine.com and notice that the lastest posts aren't the ones you're seeing in your Livemarks feed.
View the source of the feed URL (above) and you'll see that they're all there but in reverse. Live marks is displaying the first 25, which are all over a month old.

Expected behavior
If there is a standard date field like <pubDate>, feeds could be displayed desc by date by Livemarks. Otherwise I guess not much to do as it's on the publisher.

Update: I wrote Smashing Magazine and they have now fixed the publish order. So they aren't a good example anymore but I have seen the issue before with Tech Crunch for example.  I don't currently have another example, but a nice feature to add to Live Marks would be to try to find and sort by a date column if one exists.

This could potentially be harder than it looks. Parsing dates in RSS feeds is kind of complex, because a lot of feeds don't follow the standard. Maybe however Date in JS is lenient enough to work. In Rust this looks somewhat involved: https://github.com/feed-rs/feed-rs/blob/master/feed-rs/src/parser/util.rs

Ok, I did a little more research and answered my own question about rss having an established format. Checked a few (fairly popular) feeds and they each had a different property for the date:
https://www.smashingmagazine.com/feed
Tue, 29 Dec 2020 08:00:00 GMT
https://www.reddit.com/.rss
2020-12-29T16:34:26+00:00
http://rss.slashdot.org/Slashdot/slashdotMain
dc:date2020-12-29T23:25:00+00:00</dc:date>
As you mention, the date formats aren't standardized either, although at least these particularly examples are all ISO and can be properly parsed with the Date constructor.
So you're probably right that it's a bit of a challenge, although other RSS readers seem to handle it. Maybe you have to give up on guessing the date field name and just  check for date like values (and hope there's only one). new Date() was able to handle the few examples I found, but it seems likely you'd want some additional logic to try and cast them into a proper date (I can barely read Rust, but it looks like they're attempting that). Well...  I'm about to get very busy so unfortunately I won't be able to put money before mouth and try to do a pull request this time around.


Update eslint and fix formatting

Fix eslint error in startup-script sample.
Fixes https://circleci.com/gh/googleapis/nodejs-compute/2294.

Update package-json.lock as the eslint rules apparently have changed.
"Error: Delete ⏎····" on the new version versus "Error: Insert ⏎····" on the old version on the same line.


ci: log all containers in a pod

there are few pods in rook-ceph namespace which contains multiple containers. Logging all the containers in the rook-ceph namespace.
Signed-off-by: Madhu Rajanna<EMAIL_ADDRESS>

Closing as dup of https://github.com/ceph/ceph-csi/pull/2088


Metadata operation metrics for FlexGroup volumes

Do we have metadata operation metrics of volume that can be exposed by harvest? Does this already support?

@charz Could you let us know which all fields are you looking for or if you know of zapi which provides this information?

I'm moving this issue to "Discussion".


Fix misspelled method name

A minor little contribution :)

Well spotted, but unfortunately this change causes an error when calling  DealingPlatform::WatchlistMethods#all.
All attribute names currently have to match the spelling used by the IG API because of the mass assignment of attributes when they are returned from the API. We could extend Model#attribute with an extra option that would allow this spelling error to be fixed, but I don't think the complexity is worth it just for one attribute. It's probably better to match the IG API naming 100% even though it's a little free with its spelling in this instance.
See http://labs.ig.com/rest-trading-api-reference/service-detail?id=209.

Ah, of course, that is why I was seeing an error when calling that on my end. Didn't realise the method names had to match the API. If that's how they named it, probably best to keep it consistent with their API, even if it's named a bit poorly on their end.
Thanks for looking at it anyway :)


Error when running Powershell script

First, apparently the default setting, at least on my install of Windows 10, is set to block any untrusted scripts. Probably something that should be put in the installations instructions. A page explaining it
After all that, when I try to run the .ps1 script, the following error comes out (file paths partially hidden):
Start-Process : This command cannot be run completely because the system cannot find all the information required.
At C:\Users\DiaBomb\Desktop\...\GZDoom\editor.ps1:31 char:11
+ $gzdoom = Start-Process -FilePath $GzdoomPath -ArgumentList $allArgs  ...
+           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Start-Process], InvalidOperationException
    + FullyQualifiedErrorId : InvalidOperationException,Microsoft.PowerShell.Commands.StartProcessCommand

Wait-Process : Cannot validate argument on parameter 'Id'. The argument is null or empty. Provide an argument that is
not null or empty, and then try the command again.
At C:\Users\DiaBomb\Desktop\Doom\GZDoom\editor.ps1:32 char:18
+ Wait-Process -Id $gzdoom.Id
+                  ~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Wait-Process], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.WaitProcessCommand

Get-Content : Cannot find path 'C:\Users\DiaBomb\Desktop\Doom\GZDoom\editor.log' because it does not exist.
At C:\Users\DiaBomb\Desktop\Doom\GZDoom\editor.ps1:35 char:19
+ foreach ($line in Get-Content $editorLogPath)
+                   ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\...Doom\editor.log:String) [Get-Content], ItemNotFoundEx
   ception
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

Remove-Item : Cannot find path 'C:\Users\...\Desktop\Doom\GZDoom\editor.log' because it does not exist.
At C:\Users\...\Desktop\Doom\GZDoom\editor.ps1:68 char:1
+ Remove-Item $editorLogPath -Force
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\...Doom\editor.log:String) [Remove-Item], ItemNotFoundEx
   ception
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand

It's a problem on my side. This error appears if you point to the gzdoom folder in the settings file and not the exe. It wasn't exactly obvious in the installation instructions if it needs just the folder or the exe.
Should I close this now?


[Enhancement] Remove warn log in Replica.updateReplicaInfo

What type of PR is this：

[ ] bug
[ ] feature
[x] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #7819
Problem Summary(Required) ：

[FE PR Coverage check]
:heart_eyes: pass : 0 / 0 (0%)

Should rebase main branch

[FE PR Coverage check]
:heart_eyes: pass : 0 / 0 (0%)


#419 again TypeError: Cannot create property 'nonce' on number '

#419 again
2016-08-22 03:40:31 (INFO):     BUY order was not (fully) filled, cancelling and creating new order
\gekko\node_modules\btc-e\btc-e.js:46
params.nonce = Math.round((new Date()).getTime() / 1000);
^
TypeError: Cannot create property 'nonce' on number '1203764527'
at BTCE.makeRequest (\gekko\node_modules\btc-e\btc-e.js:46:18)
at BTCE.orderList (\gekko\node_modules\btc-e\btc-e.js:126:8)
at bound [as orderList] \gekko\node_modules\lodash\dist\lodash.js:729:21)
at Trader.cancelOrder (\gekko\exchanges\btce.js:146:13)
at bound [as cancelOrder] (\gekko\node_modules\lodash\dist\lodash.js:729:21)
at finish (\gekko\core\portfolioManager.js:282:21)
at bound (\gekko\node_modules\lodash\dist\lodash.js:729:21)
at check (\gekko\exchanges\btce.js:137:7)
at bound (\node_modules\lodash\dist\lodash.js:729:21)
at Request._callback \gekko\node_modules\btc-e\btc-e

Again :
2016-08-24 05:00:36 (INFO):     attempting to BUY 4.350781412632024 ETH at BTC-E
2016-08-24 05:01:38 (INFO):     BUY order was not (fully) filled, cancelling and
creating new order
C:\Users\Cabanou\Trading\gekko\node_modules\btc-e\btc-e.js:46
params.nonce = Math.round((new Date()).getTime() / 1000);
^
TypeError: Cannot create property 'nonce' on number '1207364211'
But :
24.08.16 06:05	Your order #1207364211 (buy 3.60029421 ETH @ 11.07946) filled completely:


stop(); and restart() keyboard navigation handlers

Hi,
Fist of all, thanks for this great framework ;)
I use impress as an original "small website" and on one step there's a portfolio which is driven by keyboard navigation too.
I'd like to know if there's a way to "stop and restart" impress().api (in fact, just the keyboard navigation handlers.) to avoid conflict. (both slide and step advance... )
Any help will be appreciated!
Best regards from france

@dfernandeza This will probably be handled on https://github.com/impress/impress.js/issues/409.


Readme: Add origKey to replicate example

Based on some errors I just got, it looks like including the original archive's key is required for replication

Ah right


unexpected 'const' (T_CONST), expecting variable (T_VARIABLE)

Hello after unzipping the CMS I have got an error.
Complete error is:
PHP Parse error:  syntax error, unexpected 'const' (T_CONST), expecting variable (T_VARIABLE) in /var/www/html/wondercms/index.php on line 14
Software versions:
Server version: Apache/2.4.25 (Debian)
Server built:   2019-10-13T15:43:54
PHP 7.0.33-0+deb9u6 (cli) (built: Oct 24 2019 18:50:20) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.0.33-0+deb9u6, Copyright (c) 1999-2017, by Zend Technologies

Hello @camillo777.
To debug this easier, please answer the following:

Which WodnerCMS version are you trying to run?
WonderCMS version 3.0.0+ requires PHP 7.2 or higher.
Do you have the following modules enabled?

mbstring extension
Zip extension
mod_rewrite module enabled

Hello @robiso here the info (all 3 yes):
Version:
WonderCMS-2.7.0.zip
php -m
[PHP Modules]
calendar
Core
ctype
curl
date
dom
exif
fileinfo
filter
ftp
gd
gettext
hash
iconv
intl
json
libxml
**mbstring**
mcrypt
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
Phar
posix
readline
Reflection
session
shmop
SimpleXML
sockets
SPL
standard
sysvmsg
sysvsem
sysvshm
tokenizer
wddx
xml
xmlreader
xmlwriter
xsl
Zend OPcache
**zip**
zlib

[Zend Modules]
Zend OPcache

apache2ctl -M
Loaded Modules:
 core_module (static)
 so_module (static)
 watchdog_module (static)
 http_module (static)
 log_config_module (static)
 logio_module (static)
 version_module (static)
 unixd_module (static)
 access_compat_module (shared)
 alias_module (shared)
 auth_basic_module (shared)
 authn_core_module (shared)
 authn_file_module (shared)
 authz_core_module (shared)
 authz_host_module (shared)
 authz_user_module (shared)
 autoindex_module (shared)
 deflate_module (shared)
 dir_module (shared)
 env_module (shared)
 filter_module (shared)
 mime_module (shared)
 mpm_prefork_module (shared)
 negotiation_module (shared)
 php7_module (shared)
 reqtimeout_module (shared)
 **rewrite_module** (shared)
 setenvif_module (shared)
 status_module (shared)

@camillo777, can you try bumping your PHP version to at least 7.1 and see if it solves the issue?
https://stackoverflow.com/a/51568547

Hello @robiso
I know this could maybe solve the issue, but I cannot upgrade due to another app on the same server which requires 7.0 by now.
Somewhere in the docs I read that wondercms 2 could be compatible with php 5.5 and up.

I think if you would just find-replace "const" to "" in index.php you'd solve this problem. I'm not sure what kind of 7.1+ features WonderCMS uses but if it's just the constants, you can just remove the const.

Closing this issue.
@camillo777 feel free to comment or reopen if necessary.

Hello @robiso @StephanStanisic
it seems I was able to solve it; removed visibility modifiers (public) from this constant line:
const MIN_PASSWORD_LENGTH = 8;
See this answer for reference:
https://stackoverflow.com/questions/51567998/parse-error-syntax-error-unexpected-const-t-const-expecting-variable-t-v
The ability to specify the visibility of class constants was only added in PHP 7.1

@camillo777 thank you for getting back with more information.
Keep in mind if you make any changes to your index.php and if you update WonderCMS via the 1-click updater, all the changes you've made to index.php will be overwritten.

I'd like to mention that with Nginx it's actually pretty easy to use 2 php versions, since multiple php versions can be installed next to each other.
When installing php, for example 7.0 and 7.2 they have their own folder with their own php.ini etc. With Nginx you can set the php version on a site basis, just by changing the virualhost config. For example: (inside the server block)
location ~ \.php$ {
       include snippets/fastcgi-php.conf;

       # With php-fpm (or other unix sockets):
       fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
}

You can change 7.0 here for any other php version installed (they are listed in /etc/php)

@robiso @StephanStanisic
Hello again, just I am wonder-ing if version of php >7 is really required for wonder CMS.
Because the CMS is so lightweight and minimal that it's adoption could be improved maybe by more pho version compatibility. Even version 3 could benefit from this if no php feature is really required. Just my idea.


Manager application

Manager application
Checks before requesting a review

[x] I have performed a self-review of my code.
[ ] If it is a core feature, I have added tests.
[x] If it is a front-end feature, I have manually tested it.
[x] I have added detailed documentation to my code.

Summary
Implementation of the front-end application for the manager that allows him

to check all the ingredient in the warehouse
to restock an ingredient
to add a new ingredient

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Fielddata is disabled on text fields by default

When I using this code:
$books = Book::search()     ->multiMatch(['title', 'description'], 'ham on rye', ['fuzziness' => 'AUTO'])     ->sortBy('date')     ->paginate();
I give a error: Fielddata is disabled on text fields by default.
Can you help me to set fielddata=true

mappings/app_models_book@map
...
Map::create($this->getModelType(),function(Blueprint $map){
    $map->date('date', [
        'fielddata' => true,
        'format'    => 'yyyy-MM-dd HH:mm:ss'
    ]);
},$this->getModelIndex());

$ php artisan mapping:rerun

Thanks. Its working for me

So i have a new issue.
I have a "updated_at" field is date type. When I run the command
 php artisan mapping:run
Then It show a message:
"Can't merge a non object mapping [updated_at] with an object mapping [updated_at]"
can u help me?

Just a tip but I think this is the solution.
$ curl -XDELETE "http://localhost:9200/plastic"
$ curl -XPUT "http://localhost:9200/plastic"
$ php artisan mapping:run

Thanks. I got it


Having problem modifying sample_distorted_bounding_box

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): source
TensorFlow version: r1.14
Python version: 3.5
Installed using virtualenv? pip? conda?: virtualenv
Bazel version (if compiling from source): 0.25.2
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: 9.0/7.0
GPU model and memory:

Describe the problem
Hi all. I am trying to modify the behavior of tf.image.sample_distorted_bounding_box so I digged into sample_distorted_bounding_box_op.cc and made some changes including several lines of cout << "Hi" << endl; for logging. Then I recompiled this library using bazel build //tensorflow/core/kernels/sample_distorted_bounding_box_op. However when I reran tf.image.sample_distorted_bounding_box, it seems nothing changed. May I know what else should I do to modify the TensorFlow built-in api?
Thanks in advance!

This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there and can provide faster and better help for such issues. Thanks!


Add VPC and IAM resources to request_quota_increase + code gen script

THIS PR IS BACKWARDS INCOMPATIBLE the existing keys, nat_gateway and nacl_rules, were changed.
This is an improvement of #38.
Changes:

New script to generate code to generate the necessary code to add a new resource
No need to use the 'Service Code' to lookup for resources, using aws_servicequotas_service_quota.
nat_gateway renamed to vpc_nat_gateways_per_availability_zone to follow new pattern
nacl_rules renamed to vpc_rules_per_network_acl to follow new pattern

@marinalimeira This is an old PR, but I like the changes. Can we fix the conflicts, merge this up and take care of NITs later?

Upgrade test results for build 417

Modules upgraded from main => add-more-resources-to-quota
Terraform upgraded from latest:^1.0 => latest:^1.2

⚠ If plan results are not all 0's, review the plan output to ensure backward compatibility.

Example
Modules to upgrade
Plan
Upgraded?
Patched
Upgraded?

join-path
instance-typejoin-path
0, 0, 0
:white_check_mark:

instance-type
instance-typejoin-path
0, 0, 0
:white_check_mark:

Upgrade test results for build 421

Modules upgraded from main => add-more-resources-to-quota
Terraform upgraded from latest:^1.0 => latest:^1.2

⚠ If plan results are not all 0's, review the plan output to ensure backward compatibility.

Example
Modules to upgrade
Plan
Upgraded?
Patched
Upgraded?

join-path
instance-typejoin-path
0, 0, 0
:white_check_mark:

instance-type
instance-typejoin-path
0, 0, 0
:white_check_mark:

Upgrade test results for build 425

Modules upgraded from main => add-more-resources-to-quota
Terraform upgraded from latest:^1.0 => latest:^1.2

⚠ If plan results are not all 0's, review the plan output to ensure backward compatibility.

Example
Modules to upgrade
Plan
Upgraded?
Patched
Upgraded?

join-path
instance-typejoin-path
0, 0, 0
:white_check_mark:

instance-type
instance-typejoin-path
0, 0, 0
:white_check_mark:

Hm, the tests are failing, I just locked the region to us-east-1...

Upgrade test results for build 431

Modules upgraded from main => add-more-resources-to-quota
Terraform upgraded from latest:^1.0 => latest:^1.2

⚠ If plan results are not all 0's, review the plan output to ensure backward compatibility.

Example
Modules to upgrade
Plan
Upgraded?
Patched
Upgraded?

join-path
instance-typejoin-path
0, 0, 0
:white_check_mark:

instance-type
instance-typejoin-path
0, 0, 0
:white_check_mark:

It is still failing but it works locally for me... I will investigate this on Monday.

Sooooo... I finally took the proper time to take a look at this. I was running the tests in the wrong branch, that's why it was working locally...
Well, there was a tiny typo in the test, it's fixed now! cb40b56 Tests will hopefully pass in this run and we can merge it!
@rhoboat can you take a final look? :)

Upgrade test results for build 436

Modules upgraded from main => add-more-resources-to-quota
Terraform upgraded from latest:^1.0 => latest:^1.2

⚠ If plan results are not all 0's, review the plan output to ensure backward compatibility.

Example
Modules to upgrade
Plan
Upgraded?
Patched
Upgraded?

join-path
instance-typejoin-path
0, 0, 0
:white_check_mark:

instance-type
instance-typejoin-path
0, 0, 0
:white_check_mark:

@marinalimeira Closing this as part of a PR clean-up initiative. Feel free to re-open if you continue to plan on working on this!


Issue: How to run GPT4All on GPU?

Issue you'd like to raise.
Is it possible at all to run Gpt4All on GPU? For example for llamacpp I see parameter n_gpu_layers, but for gpt4all.py - not.

Found opened ticket https://github.com/nomic-ai/gpt4all/issues/835 - GPT4ALL doesn't support Gpu yet. So, langchain can't do it also. I will close this ticket and waiting for implementation from GPT4ALL.

Thank you, is there any update on GPT4All run on GPU


haskellPackages: update stackage and hackage

This Merge
This PR is the regular merge of the haskell-updates branch into master.
This branch is being continually built and tested by hydra at https://hydra.nixos.org/jobset/nixpkgs/haskell-updates. You may be able to find an up-to-date Hydra build report at cdepillabout/nix-haskell-updates-status.
We roughly aim to merge these haskell-updates PRs at least once every two weeks. See the @NixOS/haskell team calendar for who is currently in charge of this branch.
haskellPackages Workflow Summary
Our workflow is currently described in pkgs/development/haskell-modules/HACKING.md.
The short version is this:

We regularly update the Stackage and Hackage pins on haskell-updates (normally at the beginning of a merge window).
The community fixes builds of Haskell packages on that branch.
We aim at at least one merge of haskell-updates into master every two weeks.
We only do the merge if the mergeable job is succeeding on hydra.
If a maintained package is still broken at the time of merge, we will only merge if the maintainer has been pinged 7 days in advance. (If you care about a Haskell package, become a maintainer!)

More information about Haskell packages in nixpkgs can be found in the nixpkgs manual.

This is the follow-up to #331380. Come to #haskell:nixos.org if you have any questions.

@ofborg eval

hydra-report.hs requires NIX_PATH=nixpkgs=. at the moment due to #338097 until the channel has advanced.


Update travis Node.js version & remove io.js

Thanks!

I can open a new issue, but should we consider using - 4 instead of - "4.0.0" to allow nvm to use the latest stable release of the 4.x series of node?

ah yeah... good call


Container: support configuring and respecting CMD

If we want users to be able to fully configure an image just as they would with a Dockerfile, I think we'll need to support CMD as well. Right now the spec only supports ENTRYPOINT.
I think this would mean the following changes to the spec (strawman proposal):
extend type Container {
  "Configures a default command for the container to run."
  withCmd(args: [String!]!): Container! # or withArgs?

  "This container after executing the specified command inside it"
  exec(args: [String!], opts: ExecOpts): Container! # note that args is optional now
}

When running a container that has a specified CMD (either from its image or from withCmd above), exec.args would override them. But if not present it would default to the CMD.

Good catch. I see two good options:

We take the opportunity to get quietly remove CMD from our API, on the grounds that it's a shameful relic of the past, and redundant with entrypoint. Those who still want to add it to their image can always rely on a Dockerfile.
We use a field name so clear and accurate that the confusion is gone. Personally my favorite is:

"Configures default arguments for future commands"
withDefaultArgs(args: [String!]): Container!

We take the opportunity to get quietly remove CMD from our API, on the grounds that it's a shameful relic of the past, and redundant with entrypoint. Those who still want to add it to their image can always rely on a Dockerfile.

I didn't know we saw it as a relic! Since they work in combination and both affect UX in different ways I never really saw it that way. I'm OK with omitting it but I wonder if others will think it's an oversight. The Docker docs and OCI spec don't mark it as deprecated/discouraged or anything.
If we decide not to support configuring it, do you think we should respect a CMD if it's in the image config and just not provide a way to set it, or ignore it entirely?
If we decide to support it, +1 to your withDefaultArgs - much clearer.

We take the opportunity to get quietly remove CMD from our API, on the grounds that it's a shameful relic of the past, and redundant with entrypoint. Those who still want to add it to their image can always rely on a Dockerfile.

I didn't know we saw it as a relic! Since they work in combination and both affect UX in different ways I never really saw it that way. I'm OK with omitting it but I wonder if others will think it's an oversight. The Docker docs and OCI spec don't mark it as deprecated/discouraged or anything.

Yeah the “shameful relic” is a highly subjective opinion based on my personal experience designing it 😅

If we decide not to support configuring it, do you think we should respect a CMD if it's in the image config and just not provide a way to set it, or ignore it entirely?

Good question, it would be weird to go out of our way to ignore it. But also weird to support it without any way to see or change it…

If we decide to support it, +1 to your withDefaultArgs - much clearer.

Based on the above I think it makes more sense to support it.

Based on this discussion, the consensus would be:
extend type Container {
  "Configures default arguments for future commands"
  withDefaultArgs(args: [String!]): Container!

  "This container after executing the specified command inside it"
  exec(args: [String!], opts: ExecOpts): Container!
}

If so, I'll work on this.

@dolanor Yep, that looks good. :+1:

works for me

Same, but I think that implies we should also add withoutEntrypoint, withoutWorkdir, withoutUser.

I believe so. I don't love it, but it's better than unclear behavior.

Follow up in: https://github.com/dagger/dagger/issues/3284


⚠ Bofrost.be has degraded performance

In fbb1a3f, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 2534 ms

Resolved: Bofrost.be performance has improved in 8c9fb80.


Allow configurable cursor timeout for console commands

See: https://groups.google.com/d/topic/symfony2/2MegxHbXPuU/discussion

Removing the milestone so this doesn't show in in the 1.0 release log.


Feat: Added Angular framework

Description
Please include a summary of the change and relevant motivation and context.

Added the Angular framework which would be used to develop the front-end application

List any dependencies that are required for this change.
"dependencies": {
"@angular/animations": "~8.2.14",
"@angular/common": "~8.2.14",
"@angular/compiler": "~8.2.14",
"@angular/core": "~8.2.14",
"@angular/forms": "~8.2.14",
"@angular/platform-browser": "~8.2.14",
"@angular/platform-browser-dynamic": "~8.2.14",
"@angular/router": "~8.2.14",
"eslint": "^6.8.0",
"eslint-config-airbnb-base": "^14.1.0",
"eslint-plugin-import": "^2.20.2",
"prettier": "^2.0.5",
"rxjs": "~6.4.0",
"tslib": "^1.10.0",
"zone.js": "~0.9.1"
},
"devDependencies": {
"@angular-devkit/build-angular": "~0.803.23",
"@angular/cli": "~8.3.23",
"@angular/compiler-cli": "~8.2.14",
"@angular/language-service": "~8.2.14",
"@types/jasmine": "~3.3.8",
"@types/jasminewd2": "~2.0.3",
"@types/node": "~8.9.4",
"codelyzer": "^5.0.0",
"jasmine-core": "^3.5.0",
"jasmine-spec-reporter": "~4.2.1",
"karma": "~4.1.0",
"karma-chrome-launcher": "~2.2.0",
"karma-coverage-istanbul-reporter": "~2.0.1",
"karma-jasmine": "~2.0.1",
"karma-jasmine-html-reporter": "^1.4.0",
"protractor": "~5.4.0",
"ts-node": "~7.0.0",
"tslint": "~5.15.0",
"typescript": "~3.5.3"
}
All pull requests should be related to open issues. Indicate the issue(s) bellow and remove this line:
Fixes No issues stated yet
How Has This Been Tested?
Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code etc.
Screenshots (if applicable, else remove this line / section)
Checklist:

[x ] My code follows the style guidelines of this project
[x ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added necessary inline code documentation
[ ] I have added tests that prove my fix is effective and that this feature works
[ ] New and existing unit tests pass locally with my changes

"dependencies": {
"@angular/animations": "~9.0.0",
"@angular/common": "~9.0.0",
"@angular/compiler": "~9.0.0",
"@angular/core": "~9.0.0",
"@angular/forms": "~9.0.0",
"@angular/platform-browser": "~9.0.0",
"@angular/platform-browser-dynamic": "~9.0.0",
"@angular/router": "~9.0.0",
"eslint": "^6.8.0",
"eslint-config-airbnb-base": "^14.1.0",
"eslint-plugin-import": "^2.20.2",
"prettier": "^2.0.5",
"rxjs": "~6.5.4",
"tslib": "^1.10.0",
"zone.js": "~0.10.2"
},
"devDependencies": {
"@angular-devkit/build-angular": "~0.900.1",
"@angular/cli": "~9.0.1",
"@angular/compiler-cli": "~9.0.0",
"@angular/language-service": "~9.0.0",
"@types/node": "^12.11.1",
"@types/jasmine": "~3.5.0",
"@types/jasminewd2": "~2.0.3",
"codelyzer": "^5.1.2",
"jasmine-core": "~3.5.0",
"jasmine-spec-reporter": "~4.2.1",
"karma": "~4.3.0",
"karma-chrome-launcher": "~3.1.0",
"karma-coverage-istanbul-reporter": "~2.1.0",
"karma-jasmine": "~2.0.1",
"karma-jasmine-html-reporter": "^1.4.2",
"protractor": "~5.4.3",
"ts-node": "~8.3.0",
"tslint": "~5.18.0",
"typescript": "~3.7.5"
}
}
5:46
it is a bit higher than the one @Arinze push on the repo for a change what do we do should i follow @Arinze own or my will be better

@dallintine what you have installed locally is angular 9. Maybe we should stick with angular 8 as angular 9 is a very recent release.
Also you don't need to install angular on your end. Wait till @ArinzeOkorji has finished setting up angular and this setup is reviewed and approved, then this pull request (PR) will be merged to the develop branch so that we have a unified starter codebase.
If you haven't cloned the repo, you can just clone the repo, run git checkout develop and you are good to go. But if you have already cloned this repo, ideally just delete it and repeat the steps mentioned above, since you have not done anything locally yet.
When we do start working and you need to sync your local repo with the source repo, you will need to run a git fetch upstream to fetch latest changes from the source repo (I am assuming you have setup a remote called upstream). Then git checkout develop to switch to the develop branch. Then run git merge upstream/develop to merge changes on the source develop branch with your local develop branch. ... I think it might be good to outline our git workflow steps on slack

@ArinzeOkorji Great job so far! I will like to point out though that these dependencies should be devDependencies. So you can just run npm install --save-dev eslint eslint-config-airbnb-base .......... to have them switched to devdependencies. It won't trigger a new download
"eslint": "^6.8.0",
"eslint-config-airbnb-base": "^14.1.0",
"eslint-plugin-import": "^2.20.2",
"prettier": "^2.0.5",

@ArinzeOkorji we may experience a couple of issues with airbnb's linting rules because it was designed for JS projects while this is a typescript project.
Please checkout the following article for how to setup eslint and prettier for typescript. You have already done some part, ignore all the sections about installing plugins for react and jsx and in .eslintrc.js. Also you may ignore the part about setting up husky and lint-staged:
https://www.robertcooper.me/using-eslint-and-prettier-in-a-typescript-project
Then install the following config to enable Airbnb support for angular. Follow the instructions on the repo to set it up:
https://github.com/efoken/eslint-config-airbnb-angular
If you have any questions please reach out. Thanks


Description of Create vs. Deploy role incorrect

Referring to "Understanding our Ansible roles" in the collection README/documentation, I think the description of the Create and Deploy role is not entirely correct. The documentation says "The deploy role is going to deploy those changes to the NDFC controller. [...] This is the role that is going to make the changes in the NDFC controller."
However, I observe the Create role already applies the changes to the NDFC controller. The Deploy role applies the changes to the switches.
https://github.com/netascode/ansible-dc-vxlan
https://galaxy.ansible.com/ui/repo/published/cisco/nac_dc_vxlan/docs/

Please find my review comments in PR #140.


Valid credentials coming up as Invalid for SMB

Describe the bug
Doing a Vulnhub box (https://www.vulnhub.com/entry/digitalworldlocal-bravery,281/) and crackmapexec showing the valid combination as invalid, however patator and smbmap works absolutely find.
To Reproduce
Steps to reproduce the behavior:

Simply download the vm from vulnhub and try running the tool
Screenshot attached as well

Expected behavior
Should show as valid credential
Crackmapexec info

OS: Kali
Version of CME: 5.2.2 (The Dark Knight)
Installed from apt or using latest release ? Installed from apt. (Tried the latest release as well, same behaviour)

Additional context

Victim operating system is Centos.

Can you try with option --local-auth at the end of the command ?

Hey, Thank you!! Yes it did work with --local-auth. Bad on my part, i should have tried that, but i like always i forgot that option :)

No problem :)


Roll engine 50c2e69daff4..32f417db0d56 (26 commits)

https://github.com/flutter/engine/compare/50c2e69daff4...32f417db0d56
git log 50c2e69daff4..32f417db0d56 --no-merges --oneline
32f417db0 Roll tonic to 077be256142ede39a271385907faadf4fcc62a4d. (flutter/engine#6541)
4ee77256c Revert "Roll Dart to 1f4dfce179c8f05c9e48759300a15e671b88cc10 (#6515)" (flutter/engine#6537)
964acafeb Roll src/third_party/skia 646d917d3c71..c6a17104ad68 (1 commits) (flutter/engine#6536)
d4bae4ca4 Roll src/third_party/skia 2b2c00f6ec36..646d917d3c71 (1 commits) (flutter/engine#6535)
ff93ccf47 Roll src/third_party/skia 681692726fc0..2b2c00f6ec36 (1 commits) (flutter/engine#6534)
a4161c895 Roll src/third_party/skia 23775a2e9736..681692726fc0 (1 commits) (flutter/engine#6532)
116072e46 Roll src/third_party/skia 7435f2553f53..23775a2e9736 (1 commits) (flutter/engine#6531)
ef0b0f6e9 Roll src/third_party/skia bc7a51e79c5b..7435f2553f53 (1 commits) (flutter/engine#6530)
f46b7b971 Roll src/third_party/skia b28db529c866..bc7a51e79c5b (1 commits) (flutter/engine#6529)
9033c3902 Roll src/third_party/skia 7e67041a1428..b28db529c866 (1 commits) (flutter/engine#6528)
e6887a412 Add missing imports for unicode/utf16.h (flutter/engine#6524)
1242f6dfe Roll src/third_party/skia d38382d060ca..7e67041a1428 (2 commits) (flutter/engine#6527)
a1bbea77c Add a no-op platform view layer. (flutter/engine#6505)
2bb3afad8 Roll src/third_party/skia 21bd60daa3f3..d38382d060ca (10 commits) (flutter/engine#6526)
75e875240 Fix the Mac embedder build (flutter/engine#6525)
436f9707b Add version check for dismissable (flutter/engine#6522)
7767c785b Provide a default GL function resolver in the embedder (flutter/engine#6523)
32841dd89 Case-insensitive matching of family names for custom fonts (flutter/engine#6519)
a9076c7e6 Roll src/third_party/skia 419709dbb167..21bd60daa3f3 (11 commits) (flutter/engine#6520)
f2e7441b5 An API for loading fonts from a buffer provided by the application (flutter/engine#6508)
05aac0f27 fix ResourceExtractor npe. (flutter/engine#6461)
cf5a2a145 Roll src/third_party/skia b27a9cf2f4a8..419709dbb167 (16 commits) (flutter/engine#6517)
84fe4a9f7 Re-revert invalid line height tests (flutter/engine#6516)
5f529566c Add support for text shadows (flutter/engine#6385)
e44c10c96 Reland "Share engine layers with the framework" (#6412) (flutter/engine#6468)
ba0449971 Roll Dart to 1f4dfce179c8f05c9e48759300a15e671b88cc10 (flutter/engine#6515)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

@liyuqian : New roll has been created by https://autoroll.skia.org/r/flutter-engine-flutter-autoroll

Trybots failed. These were the failed builds: https://cirrus-ci.com/task/5856895633981440

Commit queue failed; closing this roll.


fix: fix fetching rc versions of vault cli

Fixes where the install script was fetching the RC version

Interesting. Can you shed more light on why? I thought that jq format was more human-friendly.

This is a module, and if a workspace does not have jq, then it will fail. Ideally, we want 0 dependencies if possible, and the module should be useable in any workspace.
@mafredri once suggested having coder util jq|curl|wget|zip|tar type of commands so that we do not depend on anything.

This is a module, and if a workspace does not have jq, then it will fail. Ideally, we want 0 dependencies if possible, and the module should be useable in any workspace.
@mafredri once suggested having coder util jq|curl|wget|zip|tar type of commands so that we do not depend on anything.

Yup, this is the way, let's reduce dependencies wherever we can 💪🏻. The coder util is a beautiful idea but we also want to make sure we don't bloat the agent binary. Most of this can be done in a very basic way using the stdlib though, so shouldn't add much weight.
Then again, maybe we don't need to bundle it in coder util, perhaps we can have module dependencies and we have one that's called coder/modules/util that contains a few basic commands, or the ability to download them for the relevant platform.

@mafredri @mtojek another option is to use
curl -s https://releases.hashicorp.com/vault/index.json | grep -o '"[0-9]\+\.[0-9]\+\.[0-9]\+"' | tr -d '"' | grep -v 'rc' | sort -V | tail -n 1
without using jq it also uses the index.json endpoint

another option is to use

I think I like the first option, curl + regular endpoint + grep. As index.json is one-line response, I would parse it without jq.


Fixed tests: Get-XpDirTreeRestoreFile & Test-DbaDatabaseOwner

Fixed the tests for ...


Add AZURE_TRACING_DISABLED support

Fixes: https://github.com/Azure/azure-sdk-for-net/issues/8352

Thank you @ShivangiReja


validate metadata on Collection.modify

closes: #744
Description of changes

Improvements & Bug fixes

Ensures metadata is correct type when calling Collection.modify()

Test plan

New unit test added to test_api.py
Tests passed locally

Documentation Changes
None

@jeffchuber on it!

Added a validator based on the typing of CollectionMetada:
CollectionMetadata = Dict[str, Any]
However, I noticed this was recently added to the types:
UpdateCollectionMetadata = UpdateMetadata
But it appears to be unused...
is there a reason CollectionMetadata is less restrictive? It would simplify the codebase to reuse the Metadata type used for metadatas

Oh whoops, got turned around a bit by recent changes to main..  reverting some changes..

Unfortunately this is quite out of date (due to no fault of the OP!) - for hygiene I am going to close this now.


Clear cache on component unmount

@kevbook Could you elaborate a little bit more on the issue?

Now that I think about it, I don't think you need this. React will take care of it.
componentWillUnmount() {
   this.cache = null;
}

@kevbook  So can we close this issue?


How to format the code for MYCODE

How should I format the code to send to ESP32_2 using the command MYCODE?

Here is an eample of code which works:
String goodCode = "\n\n\nimport machine\nimport utime\n\ngpio0 = machine.Pin(0, machine.Pin.OUT)\ngpio46 = machine.Pin(46, machine.Pin.OUT)\n\n\nutime.sleep(1)\n\ngpio46.value(0)\n\ngpio0.value(1)";