FileDescriptor creation do not respect Group Write permissions

When open file descriptor for write and set option .create created file does not have group write permission. Checked with all group permissions .groupWrite, .groupReadWrite, .groupReadWriteExecute
In case below, expect to have permissions 0020, but get 0000
FileDescriptor.open(filePath,
                                  .writeOnly,
                                  options: [.create],
                                  permissions: [.groupWrite])

System: MacOS 10.15.7
Swift toolchain: Swift Development Snapshot 2021-02-16
SwiftPackage revision 2e9c1a71185c828416751283b40697725da550b6

Weird. I opened https://github.com/apple/swift-system/pull/27 to ensure that we are properly mapping to the right open syscall (and I verified with a breakpoint as well).
At this point, it would be an issue with how the OS handles the open syscall. I'll figure out what the next step is for creating an issue against the OS. Could you provide precise context on what you were doing? E.g., what was the state of the filesystem prior to and after the call?

This is how UNIX works and expected assuming that your umask has the 0o20 bit set which is likely because the default umask is 0o022. It masks the write permissions for group/other out unless you change umask.
This is also documented. See for example here

          The effective mode is modified by the process's umask in
         the usual way: in the absence of a default ACL, the mode
        of the created file is (mode & ~umask).

Darwin's man 2 open has to say

The oflag argument may indicate that the file is to be created if it does not exist (by specifying the O_CREAT flag).  In this

case, open() and openat() require an additional argument mode_t mode; the file is created with mode mode as described in
chmod(2) and modified by the process' umask value (see umask(2)).

If you want to actually want permissions to be the effective mode, you'd need to chmod too. Maybe permissions isn't the best name for the argument in open? Should probably be mode. Especially given that not only permissions are set with this argument but also stuff setuid/setgid.

@weissi, now it makes more sense. I can also check tomorrow how S_IWOTH works to confirm the behavior.
@milseman I really just call code from my message and then check file status. The file did not exist before I call it and created it in the tmp folder.

This is how UNIX works and expected assuming that your umask has the 0o20 bit set which is likely because the default umask is 0o022

Well, this seems like pretty strong rationale to add umask support to System in the very near term, alongside chmod.

Maybe permissions isn't the best name for the argument in open? Should probably be mode.

I would really like to avoid the very generic term "mode" if at all possible. If I didn't know the convention and the C mode_t type, I would assume "mode" referred to read vs write ("access mode"), or even append vs replace. We should definitely call this out in the docs and probably consider a more descriptive argument label ("requestedPermissions"?).
IIUC, the argument is a requested file permissions, from which umask bits are subtracted. Since FilePermissions is an OptionSet (and thus SetAlgebra), there should be ways to specify and model this. Is there any way to query this mask without overwriting it? Otherwise, we may only be able to do something like this:
extension FilePermissions {

  /// Set the process's file mode creation mask to `restricted`, which turns off
  /// any present options during calls to `open`, `mkdir`, `mkfifo`, or `mknod`.
  ///
  /// Returns the previous value of the file mode mask.
  ///
  /// The corresponding C function is `umask`.
  public static func setFileCreationMask(
    removing restricted: FilePermissions
  ) -> FilePermissions {
    FilePermissions(rawValue: umask(restricted.rawValue))
  }
}

If we had a Process type or some such, it could be hosted there instead. Off the cuff, having it be an instance method on FilePermissions would feel weird, so it is static.

Especially given that not only permissions are set with this argument but also stuff setuid/setgid.

Is the argument that type type FilePermissions also needs to be changed? I do agree that the other bits (e.g. saveText) are oddballs, but I'm not sure a different name would be better.

This is how UNIX works and expected assuming that your umask has the 0o20 bit set which is likely because the default umask is 0o022

Well, this seems like pretty strong rationale to add umask support to System in the very near term, alongside chmod.

Hmm, I mean you're typically not supposed to override umask unless the user truly asked you to set a specific mode.

Maybe permissions isn't the best name for the argument in open? Should probably be mode.

I would really like to avoid the very generic term "mode" if at all possible. If I didn't know the convention and the C mode_t type, I would assume "mode" referred to read vs write ("access mode"), or even append vs replace. We should definitely call this out in the docs and probably consider a more descriptive argument label ("requestedPermissions"?).

Sounds good! What about setuid/setgid/sticky bit and friends though? They're not permissions but can still be used in open's mode argument.

IIUC, the argument is a requested file permissions, from which umask bits are subtracted.

Sort of, I think they are mostly subtracted but I think setuid/setgid also also typically subtracted on top. But the sticky bit usually isn't. I'm not sure anymore, has been a long time since I looked into that but the bottom line is that I think there is extra magic going on in the kernel after subtracting umask.

Since FilePermissions is an OptionSet (and thus SetAlgebra), there should be ways to specify and model this. Is there any way to query this mask without overwriting it?

Nope, you'd need to do let oldMask = umask(0); umask(oldMask) to just read it (by writing the old value back). Docs here

It is impossible to use umask() to fetch a process's umask
without at the same time changing it.  A second call to umask()
would then be needed to restore the umask.  The nonatomicity of
these two steps provides the potential for races in multithreaded
programs.
Since Linux 4.7, the umask of any process can be viewed via the
Umask field of /proc/[pid]/status.  Inspecting this field in
/proc/self/status allows a process to retrieve its umask without
at the same time changing it.

Otherwise, we may only be able to do something like this:
extension FilePermissions {

  /// Set the process's file mode creation mask to `restricted`, which turns off
  /// any present options during calls to `open`, `mkdir`, `mkfifo`, or `mknod`.
  ///
  /// Returns the previous value of the file mode mask.
  ///
  /// The corresponding C function is `umask`.
  public static func setFileCreationMask(
    removing restricted: FilePermissions
  ) -> FilePermissions {
    FilePermissions(rawValue: umask(restricted.rawValue))
  }
}

If we had a Process type or some such, it could be hosted there instead. Off the cuff, having it be an instance method on FilePermissions would feel weird, so it is static.'

Yes, definitely feels better on  Process because it affects the whole process.

Especially given that not only permissions are set with this argument but also stuff setuid/setgid.

Is the argument that type type FilePermissions also needs to be changed? I do agree that the other bits (e.g. saveText) are oddballs, but I'm not sure a different name would be better.

Good question if I'm arguing that or not. But the "sticky bit" or setuid/gid don't feel like "permissions" to me but maybe you could see them as such. For posterity, here are the docs for S_ISVTX which is commonly referred to as the "sticky bit"

The directory containing pathname has the sticky bit
(S_ISVTX) set and the process's effective UID is neither
the UID of the file to be deleted nor that of the
directory containing it, and the process is not privileged
(Linux: does not have the CAP_FOWNER capability).

changing the umask is really very rare

The fact that we cannot atomically fetch umask is worrying. If a process changing it is very rare, then we can at least provide:
extension FilePermissions {
  /// The default process-wide file mode creation mask,, which turns off
  /// any present options during calls to `open`, `mkdir`, `mkfifo`, or `mknod`.
  ///
  /// The file mode creation mask can be changed with (TODO: `setFileCreationMask` or `umask`?)
  ///
  /// The corresponding C function is `umask`.
  public static var defaultFileCreationMask(
    removing restricted: FilePermissions
  ) -> FilePermissions {
    FilePermissions(rawValue: umask(restricted.rawValue))
  }
}

Since it's not affecting anything, a static member on FilePermissions makes sense. Having a decl here also makes it easier to refer to from open's docs.

They don't feel like permissions (to me) ... Good question if I'm arguing that or not. But the "sticky bit" or setuid/gid don't feel like "permissions" to me

Hmm. From first principals, we want a RawRepresentable struct + OptionSet which provides a Swifty type-safe alternative to the weakly typed mode_t and or-ing global constants. We're looking at designing the interface to the type but not its core function or capabilities. It's fairly common that extra stuff gets retrofitted onto unused bits for these types. We care about the global namespace and this pertains to files, so the most general name we'd want to go with would probably be FileMode. I don't think we'd go with FileDescriptor.Mode as it's not really specific to descriptors the way access modes or cursor positions are.
I don't think the name FilePermissions is actively bad/harmful. But it is removed enough from mode_t that it might not be obvious that this is just the low-level wrapper type and not some higher-level library representation of the concept of permissions (which could be more nuanced than mode, e.g. ACL lists).
The best opportunity or argument we may have for a rename would be with Window support. @compnerd, comments on this?
IIUC, Windows has file attributes (File.GetAttributes), which is like the result of stat's S_IFMT (yet another thing packed into mode_t sometimes). The only real overlap WRT permissions is whether a file is read-only, and _wsopen_s can take the S_IWRITE option to make a newly created file writable (all files are readable). An alternate name could be FileAttributes (i.e. go with the Windows name), but I don't want to introduce confusion with chattr and xattr.
When we get around to providing stat/File.GetAttributes, we'd probably provide the result as a RawRepresentable struct and might want to use such a FileMode type for the corresponding field. mkdir produces undefined results if bits other than the bottom 9 are used, while mknod uses them. I'm uncertain what open does with the extra bits... We could definitely use some more type safety here. We could add a FileMode type which can be used for this purpose, and deprecate anything non-permissions-y inside FilePermissions, treating it as a good time to also fix the Windows permissions story.
Rust went with fs::Permissions, which wraps the mode/attributes, but provides only the intersection of the platforms as API (readOnly). Rust provides fs::set_permissions which will invoke chmod/SetFileAttributes.

System calls its version of mode_t FilePermissions. If that's not the right name, that's unfortunate; but I don't think changing it would be a good idea at this point.
There is no expectation that mode_t would translate to Windows SDK APIs -- the Windows port ought to be able to lovingly import the corresponding concepts as true to their nature as possible.

S_IFMT's packed bits cannot be modeled well with an OptionSet like FilePermissions anyway -- I think System's wrapper struct for struct stat just needs to model the file format with a standalone property.


Made changes to the coinflip.py file

Type of change

[ ] Added new project
[ ] Bug fix
[x] New features
[x] Enhanced documentation

Changes proposed in this pull request

Project name : Coin Flip

Short description :  Added more functionality to the "answer_y" user input and removed a print statement

Checklist

[x] The code is Python 3 compatible
[x] I have added relevant comments in my file to make code understandable
[ ] I have added my project in a folder which includes all the necessary files

Thanks for the changes ⭐


Autogenerated 'tailwind.config.js' isn't compatible with workspace paths

Hi, and thanks for the Leptos - learning it and enjoying the rust full-stack idea!
So i went with start-axum-workspace template, added tailwind-input-file = "style/tailwind.css" to workspace Cargo.toml and then spent some debugging and wondering why no styles from app/src/lib.rs are in the output css, when main.scss and tailwind.css are clearly being processed.
I don't have much experience with Tailwind and its configuration file, which certainly added to my confusion.
Took me a while to realize that a ./src/**/*.rs path in module.exports.content.files in 'tailwind.config.js' won't actually process any Rust files, since we're in a workspace without ./src dir. After changing it to ./app/src/**/*.rs - everything works as intended now.
So the question is - should this be fixed, and if so - what would the fix be:

change generated version to ./*/src/**/*.rs
try to detect that we're in workspace and add ./app/src/**/*.rs if that's the case
something else?

I can try and add PR, but this needs feedback with decision, and maybe some guidance.

Just came across this, thanks for flaggint this haha

The tailwind cli isn't autogenerated, it's a static file in the start-axum-workspace template. So I'll move this issue there. A PR to fix this would be most welcome


[Bug]: Opens multiple dialog box and UI fixes.

Give a brief about the bug ✍
Description

The buttons are not interactive on hover.

When the two buttons {setting and ? } are pressed one after other, both the dialog boxes are visible and which is not desired.
The pause button becomes invisible when it is pressed so color combination can be better
I would like to work on this issue under GSSoC'24.

What is the expected behavior? 🤔
Description
The fighters can be selected based on better UI experience for the user to see the responsiveness of the game .
The dialog boxes should close when another option is clicked or when they are re-clicked.
The pause button becomes invisible on the orange color.
Provide step by step information reproduce the bug 📄
Description

Add hover animation to the cards that when it is hovered over it glows to display what starship is being selected also change the cursor hover animation to give a better interface when selecting on the page. I can create hover animations to match the theme and make it interactive for the users.  and keep the classic essence same.
Also can make the alert status panel wider for better visual appeal.
The dialog boxes should close when they are clicked again or when other option is clicked.
The pause button could have a better color to still make it visible for the users

Hey I am really eager to contribute to this game and make it a better experience for the player and I would love to work for this under GSSoC'24.
Kindly assign this issue to me.
Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

If you have any doubt you can directly send message on our discord (https://discord.gg/rZb46cCMmK), we have created seprate chennals for all projects...
Thank you :)

can you please assign me under GSSOC.'24 .

Can you please assign me this issue under GSSoC'24

The issue is fixed at #70


Add implementation of DT Apis

an mvn install locally will highlight javadoc errors and other bug checks that the pipeline has


Change Implicit Arguments to Arguments

The Implicit Arguments command is deprecated, and Arguments is already being used elsewhere.
These should be the last instances of the command in math-classes.
See also coq/coq#6802

Thanks!


Code blocks that contain shell output have misleading syntax highlighting

Code of Conduct

[x] I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?
https://docs.github.com/en/authentication/troubleshooting-ssh/using-ssh-over-the-https-port
What part(s) of the article would you like to see updated?
The code blocks containing shell output unintentionally highlight the command's output due to the presence of characters like quotes.
This creates a visually misleading representation of the actual command output.
Ideally, only the command line should be highlighted, though leaving the entire block unhighlighted is also a viable solution.
Additional information

@TheOne04 Thank you for raising this issue! I'll get this triaged for review :sparkles: Our team will provide feedback regarding the best next steps for this issue - thanks for your patience! 💛

@TheOne04 Thanks for raising this as an issue first before raising a PR, so we can discuss it: it's much appreciated.
I agree with you and I think the simplest solution is simply to remove the caret characters, as they don't serve any real purpose and the $ should mark what's a command line command as opposed to what's a prompt.
We could apply that solution to a large number of articles in our docs, and not just the page you've highlighted, so I'll defer to @nguyenalex836 on the best course of action, but we will open this up to contributors in some form.
Thank you!

@subatoi Thank you for the review and guidance here! ✨
@TheOne04 Thanks again for raising an issue! Per @subatoi's guidance, would you be open to:

Raising a PR (and linking to this issue) to remove the angle bracket characters for the code blocks you referenced in this issue
Opening a new issue for any other code blocks where the angle bracket characters should be removed

Once that is complete, we can comb through the docs on our end to find other code blocks where the angle bracket characters should be removed, and open up issues for contributors to work on 💛

@TheOne04 Thanks for raising this as an issue first before raising a PR, so we can discuss it: it's much appreciated.
I agree with you and I think the simplest solution is simply to remove the angle bracket characters, as they don't serve any real purpose and the $ should mark what's a command line command as opposed to what's a prompt.
We could apply that solution to a large number of articles in our docs, and not just the page you've highlighted, so I'll defer to @nguyenalex836 on the best course of action, but we will open this up to contributors in some form.
Thank you!

Thank you for the detailed response. While I agree with removing the > characters, this doesn't address the issue of command output being colored differently (blue) after a single quote '. This coloring is misleading, as the text is plain output, not shell code.

@TheOne04 Thank you for the elaboration! I'll let @subatoi responding regarding your last comment 💛

You're quite right @TheOne04: let's change the environment to bash and use # comments instead of the angle bracket, so:
```shell
$ ssh -T -p 443<EMAIL_ADDRESS>> Hi USERNAME! You've successfully authenticated, but GitHub does not
> provide shell access.

Will become:

$ ssh -T -p 443<EMAIL_ADDRESS># Hi USERNAME! You've successfully authenticated, but GitHub does not
# provide shell access.

If you haven't already, please refrain from making the issue that we suggested you create (cc @nguyenalex836, my apologies for creating confusion) — we'll go ahead and create an issue internally.

For now, I'll go ahead and open it up to contributors that anyone can raise a PR with the fix I've suggested above, for the article https://docs.github.com/en/authentication/troubleshooting-ssh/using-ssh-over-the-https-port

Thanks! If it's alright can I go ahead and submit a PR with the correction?
Also I would be open to help with correcting any other pages that face a similar issue but bash may not necessarily be the best solution for, so please do let me know if there's anything. :)

Oops sorry @TheOne04, I will close my PR.

My bad I guess I should've just created a PR to avoid this confusion sorry @gonmmarques


Fix regexes for .ca

I was unable to get results trying to query any .ca domains. After some debugging I found that the regexes set for .ca weren't matching the results I was getting on the command line from whois. I've tested a few hundred .ca domains with this change and they seem to populate properly.
I'm on Ubuntu 20.04 using whois 5.5.6.

most excellent , thanks


:bug: Fix retain cycle in details view

Closes #931

do you still have this problem with the changes from #932?

This PR now fixes only the retain cycle issue


Add pipelined muliplier PoC

Adding a multiplier with a truly pipelined interface. This interface currently only works for the latency-sensitive case but we can extend it to latency-insensitive too.
The key idea is that the multiplier immediately sets its read_done signal when an input is provided allowing the group to exit. The pipeline stages execute in parallel and starting the second iteration, get the correct values (the mult is 1-cycle latency).
To enable the group to execute every cycle, we need to hack around some stuff:
% fud e ex.futil --to dat -s verilog.data data.json -s verilog.cycle_limit 50 -s futil.flags '-p unroll-bound -p validate -p infer-static-timing -p top-down-st -p compile -p lower'
{
  "cycles": 10,
  "memories": {
    "left": [
      77,
      15,
      3,
      22,
      81,
      80,
      43,
      70,
      47,
      69
    ],
    "out": [
      0,
      5775,
      1245,
      99,
      2046,
      2430,
      7520,
      1849,
      770,
      2820
    ],
    "right": [
      75,
      83,
      33,
      93,
      30,
      94,
      43,
      11,
      60,
      96
    ]
  }
}

The unroll-bound eliminates the while loop completely allowing the broken top-down-st pass to do its thing. Lot to improve here but this is a good, minimal example that top-down-st should support without unroll-bound once fixed.

CC @andrewb1999

Looks great!


knife bootstrap winrm fails to bootstrap windows node

Description
bootstrap fails with " FATAL: Errno::EINVAL: Invalid argument @ dir_s_mkdir -"
command : knife bootstrap --channel current --bootstrap-version 16.1.15 -o winrm <IP_ADDRESS> -x  -P <'password'> -N
Chef Version
Chef Workstation version: 0.18.3
Chef Infra Client version: 15.10.12
Platform Version
Client -> Windows 2012 R2
Worstation -> Windows 10
Replication Case
knife bootstrap --channel current --bootstrap-version 16.1.15 -o winrm <IP_ADDRESS> -x  -P <'password'> -N
Client Output
[ip.ip.ip.ip] Detected Windows Version 6.3 Build 9600
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator\Documents>goto Version6.3
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator\Documents>goto architecture_select
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator\Documents>IF "AMD64" == "x86" IF not defined PROCESSOR_ARCHITEW6432
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator\Documents>goto chef_installed
[ip.ip.ip.ip] Checking for existing Chef Infra Client installation
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator\Documents>WHERE chef-client  1>nul 2>nul
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator\Documents>If !ERRORLEVEL! == 0 (
[ip.ip.ip.ip]
[ip.ip.ip.ip]  goto key_create
[ip.ip.ip.ip] )  else (
[ip.ip.ip.ip]
[ip.ip.ip.ip]  goto install
[ip.ip.ip.ip] )
[ip.ip.ip.ip] Existing Chef Infra Client installation detected, skipping download
[54.172.18
1.219] Writing validation key...
[ip.ip.ip.ip] Validation key written.
[ip.ip.ip.ip]
[i
[ip.ip.ip.ip] C:\Users\Administrator\Documents>(echo.{"run_list":[]}) 1>C:\chef\first-boot.json
[ip.ip.ip.ip] Starting chef-client to bootstrap the node...
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator\Documents>SET "PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\ruby\bin;C:\opscode\chef\chef\bin;C:\opscode\chef\chef\embedded\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\opscode\chef\bin"
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator
Documents>chef-client -c C:\chef/client.rb -j C:\chef/first-boot.json
[ip.ip.ip.ip] [2020-05-27T17:03:11+00:00] FATAL: Errno::EINVAL: Invalid argument @ dir_s_mkdir - C:/Users/Administrator/Documents
Aha! Link: https://chef.aha.io/features/SH-113

any help would appreciated?

@Trupti-bhansali update your chef-workstation to current release https://downloads.chef.io/chef-workstation/current/20.5.53 and remove this --channel current --bootstrap-version 16.1.15 from bootstrap command and try.. Looks like current release of chef-workstation this has been fixed https://github.com/chef/chef/issues/9844

@Vasu1105 I am using latest workstation 20.5.53.

chef -v
Chef Workstation version: 20.5.53
Chef Infra Client version: 16.1.16
Chef InSpec version: 4.18.114
Chef CLI version: 3.0.1
Test Kitchen version: 2.5.1
Cookstyle version: 6.6.9

But still getting same error:
[ip.ip.ip.ip] C:\Users\Administrator\Documents>(
[ip.ip.ip.ip] echo.chef_server_url  "https://newchef.messagehub.rd.elliemae.io/organizations/newchef"
[ip.ip.ip.ip]  echo.validation_client_name "newchef-validator"
[ip.ip.ip.ip]  echo.file_cache_path   "C:\chef/cache"
[ip.ip.ip.ip]  echo.file_backup_path  "C:\chef/backup"
[ip.ip.ip.ip]  echo.cache_options     ({:path => "C:\chef/cache/checksums", :skip_expires => true})
[ip.ip.ip.ip]  echo.chef_license "accept"
[ip.ip.ip.ip]  echo.node_name "TruptiTestNode"
[ip.ip.ip.ip]  echo.log_level        :auto
[ip.ip.ip.ip]  echo.log_location       STDOUT
[ip.ip.ip.ip]  echo.trusted_certs_dir "C:\chef/trusted_certs"
[ip.ip.ip.ip] ) 1>C:\chef\client.rb
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator\Documents>(echo.{"run_list":[]}) 1>C:\chef\first-boot.json
[ip.ip.ip.ip] Starting chef-client to bootstrap the node...
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator\Documents>SET "PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\ruby\bin;C:\opscode\chef\bin;C:\opscode\chef\embedded\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0"
[ip.ip.ip.ip]
[ip.ip.ip.ip] C:\Users\Administrator\Documents>chef-client -c C:\chef/client.rb -j C:\chef/first-boot.json
[ip.ip.ip.ip] [2020-05-29T09:43:22+00:00] FATAL: Errno::EINVAL: Invalid argument @ dir_s_mkdir - C:/Users/Administrator/Documentsef
ERROR: The following error occurred on ip.ip:

@Vasu1105 Any help would be appreciated!!

We've also encountered the same issue...
Any help would be highly appreciated!!

I am having the exact same symptoms.  When I upgraded my workstation from 0.17.5 to 0.18.3 (then later 20.6.62), I started getting those exact same errors while bootstrapping a node.  Below is the boostrap command I've been using that works perfectly on 0.17.5 that now throws errors.
knife bootstrap -o winrm xxx.domain.local --winrm-user user --winrm-password 'pass' --node-name xxx.domain.local --policy-group xxx--policy-name yyyy
[xxx.domain.local] C:\Users\xxxx\Documents>chef-client -c C:\chef/client.rb -j C:\chef/first-boot.json
[xxx.domain.local] [2020-06-02T14:28:54-05:00] FATAL: Errno::EINVAL: Invalid argument @ dir_s_mkdir - C:/Users/xxxx/Documentsef
ERROR: The following error occurred on xxx.domain.local:
ERROR:

same here:
Chef Workstation version: 20.6.62
Chef Infra Client version: 16.1.16
Chef InSpec version: 4.19.0
Chef CLI version: 3.0.4
Test Kitchen version: 2.5.1
Cookstyle version: 6.7.3

Error is:
[MLVTAREA32] C:\Users\hl_tfsbuild\Documents>chef-client -c C:\chef/client.rb -j C:\chef/first-boot.json -E MTP_QA
 [MLVTAREA32] [2020-06-11T14:50:39-05:00] FATAL: Errno::EINVAL: Invalid argument @ dir_s_mkdir - C:ef
ERROR: The following error occurred on MLVTAREA32:
ERROR:

look, the error is right there - it's not being escaped properly:
C:\chef/first-boot.json     --> Invalid argument @ dir_s_mkdir - C:ef

I've tried to reproduce this on master and I'm unable to get a failure. Master does have quite a few improvements that aren't yet in Workstation but we expect to have a release in of Chef Infra Client itself this week and ideally Workstation the week after.
Here's my test output:
bundle exec knife bootstrap <IP_ADDRESS> --connection-protocol winrm --user Administrator
Connecting to <IP_ADDRESS> using winrm
WARNING: Failed to authenticate  to <IP_ADDRESS> - trying password auth
Enter password for @<IP_ADDRESS>:

Connecting to <IP_ADDRESS> using winrm
WARNING: Performing legacy client registration with the validation key at /Users/tsmith/.chef/tim_smith-validator.pem...
WARNING: Remove the key file or remove the 'validation_key' configuration option from your config.rb (knife.rb) to use more secure user credentials for client registration.
Bootstrapping <IP_ADDRESS>
 [<IP_ADDRESS>] Checking for existing directory "C:\chef"...
 [<IP_ADDRESS>] Existing directory found, skipping creation.
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>(
 [<IP_ADDRESS>] echo.url = WScript.Arguments.Named("url")
 [<IP_ADDRESS>]  echo.path = WScript.Arguments.Named("path")
 [<IP_ADDRESS>]  echo.proxy = null
 [<IP_ADDRESS>]  echo.'* Vaguely attempt to handle file:// scheme urls by url unescaping and switching all
 [<IP_ADDRESS>]  echo.'* / into .  Also assume that file:/// is a local absolute path and that file://<foo>
 [<IP_ADDRESS>]  echo.'* is possibly a network file path.
 [<IP_ADDRESS>]  echo.If InStr(url, "file://") = 1 Then
 [<IP_ADDRESS>]  echo.url = Unescape(url)
 [<IP_ADDRESS>]  echo.If InStr(url, "file:///") = 1 Then
 [<IP_ADDRESS>]  echo.sourcePath = Mid(url, Len("file:///") + 1)
 [<IP_ADDRESS>]  echo.Else
 [<IP_ADDRESS>]  echo.sourcePath = Mid(url, Len("file:") + 1)
 [<IP_ADDRESS>]  echo.End If
 [<IP_ADDRESS>]  echo.sourcePath = Replace(sourcePath, "/", "\")
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.Set objFSO = CreateObject("Scripting.FileSystemObject")
 [<IP_ADDRESS>]  echo.If objFSO.Fileexists(path) Then objFSO.DeleteFile path
 [<IP_ADDRESS>]  echo.objFSO.CopyFile sourcePath, path, true
 [<IP_ADDRESS>]  echo.Set objFSO = Nothing
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.Else
 [<IP_ADDRESS>]  echo.Set objXMLHTTP = CreateObject("MSXML2.ServerXMLHTTP")
 [<IP_ADDRESS>]  echo.Set wshShell = CreateObject( "WScript.Shell" )
 [<IP_ADDRESS>]  echo.Set objUserVariables = wshShell.Environment("USER")
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.rem http proxy is optional
 [<IP_ADDRESS>]  echo.rem attempt to read from HTTP_PROXY env var first
 [<IP_ADDRESS>]  echo.On Error Resume Next
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.If NOT (objUserVariables("HTTP_PROXY") = "") Then
 [<IP_ADDRESS>]  echo.proxy = objUserVariables("HTTP_PROXY")
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.rem fall back to named arg
 [<IP_ADDRESS>]  echo.ElseIf NOT (WScript.Arguments.Named("proxy") = "") Then
 [<IP_ADDRESS>]  echo.proxy = WScript.Arguments.Named("proxy")
 [<IP_ADDRESS>]  echo.End If
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.If NOT isNull(proxy) Then
 [<IP_ADDRESS>]  echo.rem setProxy method is only available on ServerXMLHTTP 6.0+
 [<IP_ADDRESS>]  echo.Set objXMLHTTP = CreateObject("MSXML2.ServerXMLHTTP.6.0")
 [<IP_ADDRESS>]  echo.objXMLHTTP.setProxy 2, proxy
 [<IP_ADDRESS>]  echo.End If
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.On Error Goto 0
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.objXMLHTTP.open "GET", url, false
 [<IP_ADDRESS>]  echo.objXMLHTTP.send()
 [<IP_ADDRESS>]  echo.If objXMLHTTP.Status = 200 Then
 [<IP_ADDRESS>]  echo.Set objADOStream = CreateObject("ADODB.Stream")
 [<IP_ADDRESS>]  echo.objADOStream.Open
 [<IP_ADDRESS>]  echo.objADOStream.Type = 1
 [<IP_ADDRESS>]  echo.objADOStream.Write objXMLHTTP.ResponseBody
 [<IP_ADDRESS>]  echo.objADOStream.Position = 0
 [<IP_ADDRESS>]  echo.Set objFSO = Createobject("Scripting.FileSystemObject")
 [<IP_ADDRESS>]  echo.If objFSO.Fileexists(path) Then objFSO.DeleteFile path
 [<IP_ADDRESS>]  echo.Set objFSO = Nothing
 [<IP_ADDRESS>]  echo.objADOStream.SaveToFile path
 [<IP_ADDRESS>]  echo.objADOStream.Close
 [<IP_ADDRESS>]  echo.Set objADOStream = Nothing
 [<IP_ADDRESS>]  echo.End If
 [<IP_ADDRESS>]  echo.Set objXMLHTTP = Nothing
 [<IP_ADDRESS>]  echo.End If
 [<IP_ADDRESS>] ) 1>C:\chef\wget.vbs
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>(
 [<IP_ADDRESS>] echo.param(
 [<IP_ADDRESS>]  echo.   [String] $remoteUrl,
 [<IP_ADDRESS>]  echo.   [String] $localPath
 [<IP_ADDRESS>]  echo.)
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.$ProxyUrl = $env:http_proxy;
 [<IP_ADDRESS>]  echo.$webClient = new-object System.Net.WebClient;
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.if ($ProxyUrl -ne '') {
 [<IP_ADDRESS>]  echo.  $WebProxy = New-Object System.Net.WebProxy($ProxyUrl,$true)
 [<IP_ADDRESS>]  echo.  $WebClient.Proxy = $WebProxy
 [<IP_ADDRESS>]  echo.}
 [<IP_ADDRESS>]  echo.
 [<IP_ADDRESS>]  echo.$webClient.DownloadFile($remoteUrl, $localPath);
 [<IP_ADDRESS>] ) 1>C:\chef\wget.ps1
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>(
 [<IP_ADDRESS>]
 [<IP_ADDRESS>]
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] )
 [<IP_ADDRESS>] Detected Windows Version 10.0 Build 17763
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>goto Version10.0
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>goto architecture_select
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>IF "AMD64" == "x86" IF not defined PROCESSOR_ARCHITEW6432
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>goto chef_installed
 [<IP_ADDRESS>] Checking for existing Chef Infra Client installation
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>WHERE chef-client  1>nul 2>nul
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>If !ERRORLEVEL! == 0 (
 [<IP_ADDRESS>]
 [<IP_ADDRESS>]  goto key_create
 [<IP_ADDRESS>] )  else (
 [<IP_ADDRESS>]
 [<IP_ADDRESS>]  goto install
 [<IP_ADDRESS>] )
 [<IP_ADDRESS>] No existing installation of Chef Infra Client detected
 [<IP_ADDRESS>] Checking for existing downloaded package at "C:\Users\ADMINI~1\AppData\Local\Temp\chef-client-latest.msi"
 [<IP_ADDRESS>] Found existing downloaded package, deleting.
 [<IP_ADDRESS>] Attempting to download client package using PowerShell if available...
 [<IP_ADDRESS>] powershell.exe -ExecutionPolicy Unrestricted -InputFormat None -NoProfile -NonInteractive -File  C:\chef\wget.ps1 "https://www.chef.io/chef/download?p=windows&pv=2016&m=x86_64&DownloadContext=PowerShell&channel=stable&v=16" "C:\Users\ADMINI~1\AppData\Local\Temp\chef-client-latest.msi"
 [<IP_ADDRESS>] Download via PowerShell succeeded.
 [<IP_ADDRESS>] Installing downloaded client package...
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>msiexec /qn /log "C:\Users\ADMINI~1\AppData\Local\Temp\chef-client-msi20619.log" /i "C:\Users\ADMINI~1\AppData\Local\Temp\chef-client-latest.msi"
 [<IP_ADDRESS>] Successfully installed Chef Infra Client package.
 [<IP_ADDRESS>] Installation completed successfully
 [<IP_ADDRESS>] Writing validation key...
 [<IP_ADDRESS>] Validation key written.
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>(
 [<IP_ADDRESS>] echo.-----BEGIN CERTIFICATE-----
 [<IP_ADDRESS>]  echo.-----END CERTIFICATE-----
 [<IP_ADDRESS>] ) 1>C:\chef/trusted_certs/34_213_52_193.crt
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>(
 [<IP_ADDRESS>] echo.chef_server_url  "https://api.chef.io/organizations/tim_smith"
 [<IP_ADDRESS>]  echo.validation_client_name "tim_smith-validator"
 [<IP_ADDRESS>]  echo.file_cache_path   "C:/chef/cache"
 [<IP_ADDRESS>]  echo.file_backup_path  "C:/chef/backup"
 [<IP_ADDRESS>]  echo.cache_options     ({:path => "C:/chef/cache/checksums", :skip_expires => true})
 [<IP_ADDRESS>]  echo.chef_license "accept"
 [<IP_ADDRESS>]  echo.# Using default node name (fqdn)
 [<IP_ADDRESS>]  echo.log_level        :auto
 [<IP_ADDRESS>]  echo.log_location       STDOUT
 [<IP_ADDRESS>]  echo.trusted_certs_dir "C:/chef/trusted_certs"
 [<IP_ADDRESS>] ) 1>C:\chef\client.rb
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>(echo.{"run_list":[]}) 1>C:\chef\first-boot.json
 [<IP_ADDRESS>] Starting chef-client to bootstrap the node...
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>SET "PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\ruby\bin;C:/opscode/chef\bin;C:/opscode/chef\embedded\bin;C:\tools\ruby27\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\Amazon\cfn-bootstrap\;C:\ProgramData\chocolatey\bin;C:\Program Files\Git\cmd;C:\Program Files\Microsoft VS Code\bin;C:\Users\Administrator\AppData\Local\Microsoft\WindowsApps"
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] C:\Users\Administrator\Documents>chef-client -c C:/chef/client.rb -j C:/chef/first-boot.json
 [<IP_ADDRESS>] Starting Chef Infra Client, version 16.1.16
 [<IP_ADDRESS>] Creating a new client identity for EC2AMAZ-3SCR1KK.us-west-2.compute.internal using the validator key.
 [<IP_ADDRESS>] resolving cookbooks for run list: []
 [<IP_ADDRESS>] Synchronizing Cookbooks:
 [<IP_ADDRESS>] Installing Cookbook Gems:
 [<IP_ADDRESS>] Compiling Cookbooks...
 [<IP_ADDRESS>] [2020-06-16T23:10:13+00:00] WARN: Node EC2AMAZ-3SCR1KK.us-west-2.compute.internal has an empty run list.
 [<IP_ADDRESS>] Converging 0 resources
 [<IP_ADDRESS>]
 [<IP_ADDRESS>] Running handlers:
 [<IP_ADDRESS>] Running handlers complete
 [<IP_ADDRESS>] Chef Infra Client finished, 0/0 resources updated in 11 seconds

My flags are a bit different since I'm using the new Chef 15+ flags to avoid deprecation warnings. Additionally, I'd avoid using -P since that displays your password. Knife will prompt you automatically for the password

@tas50 this issue is getting reproduced on Windows---->Windows bootstrap process. From your results, it seems you are trying to bootstrap from Linux--->Windows. Could you please check once?
Thank you!!

@tas have you been able to take another look at this? it's a blocker for us.

@mcascone This is being worked on now

Having same issue, any solution yet?

Hello, is this fixed. It is a big problem. Which chef-client version stable that can we use in the meantime?

For reference https://github.com/chef/chef/pull/10141#issuecomment-665080902

I keep getting a notification to update to 20.8.111, but it doesn't look like this is fixed yet?

 [2020-08-14T11:30:20+00:00] FATAL: Errno::EINVAL: Invalid argument @ dir_s_mkdir - C:/Users/chef/Documentsef
ERROR: The following error occurred on chef-lab-hcniui.westus.cloudapp.azure.com:
ERROR: cmd.exe : Exception calling "DownloadFile" with "2" argument(s): "The request was
         + CategoryInfo          : NotSpecified: (Exception calli...he request was :String) [], RemoteException
         + FullyQualifiedErrorId : NativeCommandError
     aborted: Could not create SSL/TLS secure channel."
     At C:\chef\wget.ps1:14 char:1
     + $webClient.DownloadFile($remoteUrl, $localPath);
     + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
         + FullyQualifiedErrorId : WebException

     Failed download: download completed, but downloaded file not found

I get this when I run
knife bootstrap -o winrm *fqdn*  -u *username*-P *password* -N windows-client --verbose
If I run it again, i get
[2020-08-14T11:57:55+00:00] FATAL: Errno::EINVAL: Invalid argument @ dir_s_mkdir - C:/Users/chef/Documentsef
ERROR: The following error occurred on ****:
ERROR:

This is being addressed in https://github.com/chef/chef/pull/10298 and I've tested of it's working for windows-to-windows bootstrap.

Using the latest client version 16.4.35. Now i get similar error
[2020-08-18T10:59:42+00:00] FATAL: Errno::EINVAL: Invalid argument @ dir_s_mkdir - C:/windows/system32ef

@aswinfrancis91 Please check if there is anything other than 16.4.35 is installed, Update your GEM_PATH, GEM_HOME, and PATH variables per v16.4.35.

I ran into this as well. I believe this is addressed in an upcoming version but even if you are using the "fixed" version, you will continue to tun into this problem if you are trying to bootstrap the same node that previously failed if that node has the client.rb file from the previous boostrap. The issue is the backslashes in that file. You should be good if you uninstall any chef client install that might be on that node along with the c:\chef directory.

I have the same experience. I have to fix the slashes kn the client.rb file manually and rerun chef-client, whether older or newest version of the client. . Which is the upcoming workstation version where this will be fixed?

This is fixed in https://github.com/chef/chef/pull/10298 and shipped in Chef Infra Client v16.4.35 and Chef workstation v20.8.125
Tested with both successfully.


How to diagnose "Running (Provisioning failed)" error.

I am creating a linux VM using python sdk. The vm is getting created with message "Running (Provisioning failed)" on portal. I am also unable to log into the VM using putty.
There are no error messages from python code. I am displaying following messages.
{'request_id': '353d9e91a77da8659703a3dd0575245c'}
{'status': u'Succeeded', 'error': None, 'id': u'', 'http_status_code': u'200'}
{'request_id': '737871f54ae8a1e49d1b0ff92503d64a'}
{'status': u'Succeeded', 'error': None, 'id': u'', 'http_status_code': u'200'}
How can I debug this? Can anyone suggest ways to display debugging information so that I can at know what is going on. Any  help would be greatly appreciated. Here is my python code.
Thanks
from azure.servicemanagement import *
import cfg.config as config
import base64
import time
SERVICE_CERT_THUMBPRINT = '970EFBE87D1F757D2F1576EC328F93BE92756404'

def main(hostname, location, image_name,storage_name,container,blob, username, password, user_public_key, user_private_key, user_certificate_pfx):
subscription_id = config.subscription_id
certificate_path = config.certificate_path

sms = ServiceManagementService(subscription_id, certificate_path)
try:
    status = sms.get_hosted_service_properties(hostname)
    print "service %s is found" %(hostname)
except:
    print "service %s is NOT found and is being created" %(hostname)
    sms.create_hosted_service(service_name=hostname, label=hostname, location=location)
try:
    sms.get_storage_account_properties(storage_name)
    print "storage %s is found" %(storage_name)
except:
    print "storage %s is NOT found and is being created" %(storage_name)
    sms.create_storage_account(storage_name, storage_name, storage_name, None, location);

windows_blob_url = "blob.core.windows.net"
media_link = "http://" + storage_name + "." + windows_blob_url + "/" + container + "/" + blob
os_hd = OSVirtualHardDisk(image_name, media_link)

pk = PublicKey(SERVICE_CERT_THUMBPRINT, 'C:/temp/gpsservice/gpsservice.pem')
pair = KeyPair(SERVICE_CERT_THUMBPRINT, 'C:/temp/gpsservice/gpsservice.key')
linux_config = LinuxConfigurationSet(hostname, username, password, False)
linux_config.ssh.public_keys.public_keys.append(pk)
linux_config.ssh.key_pairs.key_pairs.append(pair)

network = ConfigurationSet()
network.configuration_set_type = 'NetworkConfiguration'
network.input_endpoints.input_endpoints.append(ConfigurationSetInputEndpoint('ssh', 'tcp', '22', '22'))

with open(user_certificate_pfx, "rb") as bfile:
    cert_data = base64.b64encode(bfile.read())

cert_format = 'pfx'
cert_password = ''
cert_res = sms.add_service_certificate(service_name=hostname,
                        data=cert_data,
                        certificate_format=cert_format,
                        password=cert_password)
print vars(cert_res)
time.sleep(32)
status = sms.get_operation_status(cert_res.request_id)
try:
    print vars(status.error)
except:
    print vars(status)
result = sms.create_virtual_machine_deployment(service_name=hostname,
    deployment_name=hostname,
    deployment_slot='production',
    label=hostname,
    role_name=hostname,
    system_config=linux_config,
    os_virtual_hard_disk=os_hd,
    network_config=network
    )
print vars(result)
time.sleep(32)
status = sms.get_operation_status(result.request_id)
try:
    print vars(status.error)
except:
    print vars(status)
print "done"

if __name__ == '__main__':
main('rk1010', 'East US','0b11de9248dd4d87b18621318e037d37__RightImage-Ubuntu-12.04-x64-v<IP_ADDRESS>',
     'rk1010storage','rkcontainer','rkblob','rk1010', 'Solapur28$','C:/temp/gpsservice/gpsservice.pub','C:/temp/gpsservice/gpsservice.key', 'C:/temp/gpsservice/gpsservice.pfx')

I have the same issue? Any progress?
Thanks

Were you able to get past this issue?
Sometimes parameters are incompatible and may not result in an immediate error, I suggest you try different parameters values and see what is going on. You could try again a get_operation_status after receiving the "Succeeded" to check if the message becomes equivalent to the one you receive in the portal.

Closing due to inactivity. Please feel free to re-open the bug if needed to continue the discussion.


Create generator-generic-ossf-slsa3-publish.yml

It's a entertainment that eliminates any interference of static controlled radio static waves.


Handle external-ip changes on Gateway node

What happened:
When Submariner is deployed in a Cluster with NAT disabled, the local endpoint of the cluster includes the external-ip (aka public-ip) of the active gateway node. The active gateway node of the cluster updates the endpoint info on the Broker which is subsequently sync'ed by all the member clusters. The member clusters then try to connect to the public-ip configured in the endpoint CRD to setup IPsec tunnels. So far, everything is fine.
In one of the OpenShift deployments on AWS, it was seen that when the external-ip is modified on the Active Gateway node of the cluster, the endpoint is properly updated with the new public-ip. However, this use-case is not handled properly in the  CableEngine.
What you expected to happen:
When the external-ip (or public-ip) of the active Gateway node is changed, tunnels should be reconfigured successfully with the new public-ip.
How to reproduce it (as minimally and precisely as possible):
Steps

Say we have two clusters - cluster1 and cluster2 on AWS
Deploy Submariner and join both the clusters.
Wait for IPsec tunnels to be setup and verify that Gateway status shows "connected" for both the clusters.
On the active gateway node of cluster1, remove the external-ip and reconfigure a new external-ip.
SubmarinerEnginePod on cluster1 notices this change and updates its EndpointCRD with the new public-ip.
The updated endpoint is pushed to the Broker cluster as well.
Cluster2 gets notified on the updated endpoint on cluster1.
Ideally it should tear down the previous IPsec tunnel and re-establish a new IPsec tunnel that points to the new public-ip of cluster1. This is not happening for the following reasons.

In step number 7 shown above, when cluster2 gets notification of updated endpoint, it queries the CableDriver for the GetActiveConnections() which returns the cable name (aka submariner-cable-cluster1-10-0-26-36). The CableName is constructed as follows - "submariner-cable--". The CableEngine InstallCable API simply checks the cable name[*] and assumes that connection is already established to that endpoint.
[*] https://github.com/submariner-io/submariner/blob/507166f7c5ab4f9d4a1eac0386f082e9973de695/pkg/cableengine/cableengine.go#L114
Anything else we need to know?:
This was seen with Strongswan Driver but since its a common code, it most likely will affect other drivers too.
Environment:
Showing information for cluster "cluster1":
COMPONENT                       REPOSITORY                                            VERSION
submariner                      quay.io/submariner                                          devel
submariner-operator             quay.io/submariner                                    devel
service-discovery               quay.io/submariner                                       devel

This issue can be addressed in multiple ways.
An easy solution AFAICT is to update the CableName to include public-ip as well here. But it can have an impact on existing brownfield deployments.

Well done @sridhargaddam
I tested this feature with Subctl 0.7 RC2 - It works great:
https://qe-jenkins-csb-skynet.cloud.paas.psi.redhat.com/job/Submariner-OSP-AWS/850/Test-Report/
Before VM reboot of the Gatewy node on nmanos-cluster-a, the Gateway public IP = <IP_ADDRESS> :
08:51:47     gateways:
08:51:47     - connections:
08:51:47       - endpoint:
08:51:47           backend: strongswan
08:51:47           cable_name: submariner-cable-nmanos-cluster-a-10-166-106-214
08:51:47           cluster_id: nmanos-cluster-a
08:51:47           hostname: ip-10-166-106-214
08:51:47           nat_enabled: true
08:51:47           private_ip: <IP_ADDRESS>
08:51:47           public_ip: <IP_ADDRESS>
08:51:47           subnets:
08:51:47           - <IP_ADDRESS>/19
08:51:47         status: connected
08:51:47         statusMessage: Connected to <IP_ADDRESS>:4501 - encryption alg=AES_GCM_16,
08:51:47           keysize=128 rekey-time=14364
08:51:47       haStatus: active
08:51:47       localEndpoint:
08:51:47         backend: strongswan
08:51:47         cable_name: submariner-cable-default-cl1-10-166-0-235
08:51:47         cluster_id: default-cl1
08:51:47         hostname: default-cl1-k7hcq-worker-g2cnf
08:51:47         nat_enabled: true
08:51:47         private_ip: <IP_ADDRESS>
08:51:47         public_ip: <IP_ADDRESS>
08:51:47         subnets:
08:51:47         - <IP_ADDRESS>/19
08:51:47       statusFailure: ""
08:51:47       version: v0.7.0-rc2-2-ga9021c5-dev
08:51:47     globalCIDR: <IP_ADDRESS>/19

After VM reboot  - Gateway on same cluster had public IP = <IP_ADDRESS> :
08:53:39   Gateways:
08:53:39     Connections:
08:53:39       Endpoint:
08:53:39         Backend:      strongswan
08:53:39         cable_name:   submariner-cable-nmanos-cluster-a-10-166-106-214
08:53:39         cluster_id:   nmanos-cluster-a
08:53:39         Hostname:     ip-10-166-106-214
08:53:39         nat_enabled:  true
08:53:39         private_ip:   <IP_ADDRESS>
08:53:39         public_ip:    <IP_ADDRESS>
08:53:39         Subnets:
08:53:39           <IP_ADDRESS>/19
08:53:39       Status:          connected
08:53:39       Status Message:  Connected to <IP_ADDRESS>:4501 - encryption alg=AES_GCM_16, keysize=128 rekey-time=13086
08:53:39     Ha Status:         active
08:53:39     Local Endpoint:
08:53:39       Backend:      strongswan
08:53:39       cable_name:   submariner-cable-default-cl1-10-166-0-235
08:53:39       cluster_id:   default-cl1
08:53:39       Hostname:     default-cl1-k7hcq-worker-g2cnf
08:53:39       nat_enabled:  true
08:53:39       private_ip:   <IP_ADDRESS>
08:53:39       public_ip:    <IP_ADDRESS>
08:53:39       Subnets:
08:53:39         <IP_ADDRESS>/19
08:53:39     Status Failure:
08:53:39     Version:         v0.7.0-rc2-2-ga9021c5-dev
08:53:39   Global CIDR:       <IP_ADDRESS>/19

@nyechiel


RFC: Medici is slow when number of records reach 30k, add indexes

The balance queries take 100 - 400 ms average. The MongoDB Cloud Performance Adviser shows these fields are good candidates for indexing:
account_path.0: 1
account_path.1: 1
account_path.2: 1
book: 1
approved: 1

The Medici v2.1 should be released and set the indexes by default.

Indexes were added in v3.0. No API changes between v2 and v3 though.
Here is the list of indexes added: https://github.com/koresar/medici/commit/274528ef5d1dae0beedca4a98dbf706808be53bd
  transactionSchema.index({ "_journal": 1 });
  transactionSchema.index({ "account_path.0": 1, "book": 1, "approved": 1 });
  transactionSchema.index({ "account_path.0": 1, "account_path.1": 1, "book": 1, "approved": 1 });
  transactionSchema.index({ "account_path.0": 1, "account_path.1": 1, "account_path.2": 1, "book": 1, "approved": 1 });

The README was updated with more details on performance: https://github.com/koresar/medici#performance


Editable subobject

When setting editable: true to an object containing fields (I tried name), the editing process it buggy.

Ah, actually it's fine. I just converted it into a string. :smile:


HTML Select component - onchange event

Apologies if this is a known issue or if I'm doing it wrong, but it looks like a bug as far as I can tell:
The problem is with detecting the change event when a HTML select component is changed.
I have a simple HTML Select component:
        <select id="config-select">
        <option id="configOptionDummy1" value="d1">Dummy1</option>
        <option id="configOptionDummy2" value="d2">Dummy2</option>
        </select>

My java code for detecting the change:
    private static HTMLSelectElement configSelect = (HTMLSelectElement) document.getElementById("config-select");
    .....
    configSelect.addEventListener("change", new EventListener() {
        @Override
        public void handleEvent(Event event) {
            System.out.println("select " + event);
        }
    });

What happens is, at runtime, when the the selection is changed, there is a javascript error of the form:
TypeError: $this.QG is undefined

What version of TeaVM do you use? Did you try to disable minification? Can you send full stack trace of non-minified program?

Wow, quick response! :)
OK, so I have now switched off minify.
I had been using firefox, but couldn't work out how to get the stack trace, so I tried with chrome instead. The error is slightly different but it still goes wrong in the same place.

Please, use TeaVM version 0.3.0-SNAPSHOT. You should first build it from source code. Simply clone project repository and run mvn clean install -DskipTests.

Have cloned and compiled latest, and set my teavm version to 0.3.0-SNAPSHOT in the pom.
But unfortunately the error remains as before.

Steve, you may try to log the event using @JavaScriptBody (TeaVM 0.2.1), see if that makes a difference.
	@JavaScriptBody(args = { "o" }, body = "console.log(o)")
	public static native void log(JSObject obj);

In 0.3.0-SNAPSHOT the annotation is changed to something else.

Logging won't help. I think I realized the reason. JSObjects are not completely normal Java objects and they have some restrictions. One is that you can't call hashCode, equals and toString on them. Here, "select " + event calls event.toString() implicitly. Try to replace it with just
System.out.println("select");

and see whether it helps.

Quite sure that it's the toString() issue.  Just jumping a step ahead to show how else to display the event object. ;)

It's quite easy:
interface Printable extends JSObject {
   @JSMethod("toString")
    String jsToString();
}
((Printable)event).jsToString();

But there's no great need to represent event as a string.

Nice work guys, it was indeed the event reference. I had only put it in for debug purposes to verify the handleEvent() method was being called.
I notice that it is also not possible to call the methods on the event object, e.g. event.getTarget()
I see what you mean about it being javascript rather than standard java object. But, not being able to access methods like this might well feel a bit strange to Java developers - I can imagine it tripping them up. Would be good to see a nicer solution for this problem if possible, but anyway, at least I know to avoid that now. Thanks for your speedy assistance with this issue!

It's strange that it's not possible to call event.getTarget(). Can you create a minimum project that reproduces this issue? Is it reproducible with 0.3.0-SNAPSHOT?
Sorry, but it's not possible to properly implement toString, hashCode and equals on JSObjects. Won't ever be possible. So I just claim for Java developers that these objects have very special meaning. They are very low-level, and I hope eventually I'll implement wrappers around them, so regular developers will rarely deal with them directly.

The example is the same as above, except I try to print out
System.out.println("select " + event.getTarget());
event.getCurrentTarget() has the same result: TypeError: $this.characters49 is undefined
Happens on 0.3.0-SNAPSHOT as well as 0.2.1.
However event.getType() actually works ok and is equal to "change" as you would expect.

That's because event.getTarget() returns a EventTarget, which is a JSObject.  So your println statement is still trying to execute toString() on a JSObject.  event.getType() returns a java String and therefore it works.


Add an auth prompt sample. Also fix minor bugs

added a sample for auth to the existing prompt project.
minor fixes (that are also done in the node code too)

I added the ability to emulate auth cards for emulator but didnt make it compulsory yet. will do in another iteration.

@swagatmishra2007 - trying to figure out how to have the automated build+tests run on this code before I merge it. Because it's submitted as an external PR, that gate hasn't been run.

i merged this in , after testing on another machine that the build succeeds. the CI build also succeeded.`


expose osm-controller version

Please describe the Improvement and/or Feature Request
Thanks to #1149, we now have the CLI version but we also need to figure out a user could get the version of osm-controller version. Should we get this information from the helm release which contains the osm-controller image tag? or should we have an endpoint on osm-controller that returns version info? What should the cli experience be to expose this info?
Scope

New Functionality

Possible use cases
As a user, I'd like to understand what version of osm-controller I am running so I have that information for debugging purposes.

@michelleN i would love to take on this task with some initial guidance. Can I pick this up?

@nitishm this is all yours if you want it!

Should we get this information from the helm release which contains the osm-controller image tag?

This would tie us to a helm based deployment model. I think the latter,

an endpoint on osm-controller that returns version info
would be the best approach in the long run.

Should we get this information from the helm release which contains the osm-controller image tag?

This would tie us to a helm based deployment model. I think the latter,

an endpoint on osm-controller that returns version info
would be the best approach in the long run.

@draychev @michelleN WDYT?

You can retrieve the information from the controller pod by port-forwarding to the controller pod via the cli. cli already does this for a few commands namely osm dashboard, osm proxy dump-config

@shashankram correct me if I am wrong. For this task I will need to add a /version route to pkg/httpserver and then use port forward from the CLI to request that endpoint.

@shashankram correct me if I am wrong. For this task I will need to add a /version route to pkg/httpserver and then use port forward from the CLI to request that endpoint.

If you go with the approach of exposing a /version endpoint on the osm-controller HTTP server then yes.

@johnsonshi worked on a similar issue that displays the version of the control plane for a given mesh by deriving the version based on the labels added to the osm-controller deployments.
$ osm mesh list

MESH NAME   NAMESPACE        CONTROLLER PODS                   VERSION   SMI SUPPORTED
osm         arc-osm-system   osm-controller-744866fdfb-xb6xv   v0.8.4    Unknown

This issue seems pretty much the same as https://github.com/openservicemesh/osm/issues/2848 which was resolved. Closing this.


[PE-D][Tester D] Id number of student cannot be seen if their names are too long

Add a new student with a very long name e.g. Ong Xiao Ming Ling Ling Christopher Prince Alexander GreenSwift Leonardo The Third

Expected:
Should be able to see all fields as per usual.
Actual:
When the name is too long, the id number of the student cannot be seen, which may cause troubles for the tutor managing the students.

Labels: severity.Medium type.FunctionalityBug
original: shavonneg/ped#5

Lack of it is not a bug - From the TA

https://github.com/nus-cs2103-AY2324S2/forum/issues/700


Overflows and No Material Widget Found errors

Environment
Testing out with example app and web3modal_flutter: ^1.2.1, Flutter 3.12.0-1.1.pre • channel beta on web chrome.
Problem
Overflow on desktop horizontal view:

No Material Widget Found errors:

Error Logs:
══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
The following assertion was thrown building _InkResponseStateWidget(gestures: [tap], mouseCursor:
null, clipped to BoxShape.rectangle, dirty, state: _InkResponseState#6bbfc):
No Material widget found.
_InkResponseStateWidget widgets require a Material widget ancestor within the closest
LookupBoundary.
In Material Design, most widgets are conceptually "printed" on a sheet of material. In Flutter's
material library, that material is represented by the Material widget. It is the Material widget
that renders ink splashes, for instance. Because of this, many material library widgets require that
there be a Material widget in the tree above them.
To introduce a Material widget, you can either directly include one, or use a widget that contains
Material itself, such as a Card, Dialog, Drawer, or Scaffold.
The specific widget that could not find a Material ancestor was:
  _InkResponseStateWidget
The ancestors of this widget were:
  ...
  InkWell
  GridListItem-[<'Zerion'>]
  RepaintBoundary
  IndexedSemantics
  _SelectionKeepAlive
  ...
 Consider wrapping it with Dialog
Wrap the container with SingleChildScrollView or add size constraints to the Container
https://github.com/WalletConnect/Web3ModalFlutter/blob/b569f62d13197cbf09c9a7db7a790d0ab2811c4d/lib/widgets/walletconnect_modal.dart#L92C5-L96C12

Please use the WalletConnectModal instead of this library, this one isn't ready for use yet, and hasn't been tested in any way.
I really appreciate the feedback tho! The usage of Dialog will likely fix the showDialog issue (Which hasn't been tested at all).
Your overflow problem should be fixed in the WalletOnnectModal

This library is deprecated for now.
Please use the WalletConnectModal.


Fix Boulder email body

Replaced approximate funding numbers with exact funding numbers, sourced
from Boulder's 2020 Budget Documentation. As a result, the logic of the
email's argument had to shift to a discussion of how budgets have grown
or shrunk since 2018, which was a strong funding year.

Does this fix an issue?

It fixes an issue with the funding numbers being reported.
Previously the email cited <$5,000,000 (which I could not corroborate from any sources) going to the city's HHS, but the figure from the most recent budget was $21,000,000.

Please the issue you are fixing by adding closes #324234244 in the main comment. ALso issues  need to be tagged with the yellow tag before us creating PRs.

I’m sorry, new to collaborating on github. Are you saying I should have opened an issue first, labeled it yellow, then opened this PR? I just opened this PR without documenting an issue...

Closes: https://github.com/defund12/defund12.org/issues/749


fix sql to get count or products

Questions
Answers

Description?
There was an issue when a product category belongs to multiple groups, it was duplicating the number of product in returned by the sql query.

Type?
bug fix

BC breaks?
no

Deprecations?
no

Fixed ticket?
Fixes https://github.com/PrestaShop/PrestaShop/issues/29043

How to test?
See the step in the parent issue

Possible impacts?
On the list of products on the wishlist.

Hello @paul-jolimoi,
Thanks for your PR. The issue is well fixed andI found no regression :tada:
https://watch.screencastify.com/v/NefjG70zReWFkSMd9O6R

Thanks @paul-jolimoi. I had the same issue of duplicated products.


Specify background cover of fold

I have a slight issue with this framework, my "cards" are all with white background but when folding or unfolding the background of the other face of the card is a wired brown! How did this happen? I mean I didn't even specific any colour! Here are some screenshots to help clarify.
I also have a bit of a problem with some constrains, it gives me the "Unable to simultaneously satisfy constraints" look, it also seems to happen in each "card":
2016-06-06 23:05:45.912 GL Dash[542:87484] Unable to simultaneously satisfy constraints.
	Probably at least one of the constraints in the following list is one you don't want.
	Try this:
		(1) look at each constraint and try to figure out which you don't expect;
		(2) find the code that added the unwanted constraint or constraints and fix it.
(
    "<NSLayoutConstraint:0x127678ec0 V:[UIView:0x1276654c0(350)]>",
    "<NSLayoutConstraint:0x1276beef0 V:[UIView:0x1276654c0(132)]>"
)

Will attempt to recover by breaking constraint
<NSLayoutConstraint:0x127678ec0 V:[UIView:0x1276654c0(350)]>

Make a symbolic breakpoint at UIViewAlertForUnsatisfiableConstraints to catch this in the debugger.
The methods in the UIConstraintBasedLayoutDebugging category on UIView listed in <UIKit/UIView.h> may also be helpful.
2016-06-06 23:05:45.914 GL Dash[542:87484] Unable to simultaneously satisfy constraints.
	Probably at least one of the constraints in the following list is one you don't want.
	Try this:

		(1) look at each constraint and try to figure out which you don't expect;
		(2) find the code that added the unwanted constraint or constraints and fix it.
(
    "<NSLayoutConstraint:0x1276beef0 V:[UIView:0x1276654c0(132)]>",
    "<NSLayoutConstraint:0x1276c2010 V:[UIView:0x1276654c0(75)]>"
)

Will attempt to recover by breaking constraint
<NSLayoutConstraint:0x1276beef0 V:[UIView:0x1276654c0(132)]>

Make a symbolic breakpoint at UIViewAlertForUnsatisfiableConstraints to catch this in the debugger.
The methods in the UIConstraintBasedLayoutDebugging category on UIView listed in <UIKit/UIView.h> may also be helpful.

Tiago

Hi tiferrie, did you have to do any additional tweaking to the code to make your unfolding animation line up properly with the views?

Sorry @jdulb17, I ended up not using this framework, was using a loads of RAM and the animations sometimes messed up the content.


Setting AZURE_CREDENTIALS as a secret breaks Github best practices

From https://docs.github.com/en/actions/security-guides/encrypted-secrets:

To help ensure that GitHub redacts your secret in logs, avoid using structured data as the values of secrets. For example, avoid creating secrets that contain JSON or encoded Git blobs.

As a result of setting AZURE_CREDENTIALS as a secret, the secret content can be displayed in the console in our workflows.
I believe the example creds: ${{ secrets.AZURE_CREDENTIALS }} should be removed from the readme and the input creds:  value should instead be built from simple (non-json) secrets or removed from the action.

Hi @arnaud-tincelin. Thank you for raising this concern. We have documented this as an alternative way to pass secrets. Please check it out.

If you want to pass Subscription ID, Tenant ID, Client ID, and Client Secret as individual parameters instead of bundling them in a single JSON object (creds) to address the security concerns for Non-OIDC login, below snippet can help with the same.
- uses: Azure/login@v1
   with:
     creds: '{"clientId":"${{ secrets.CLIENT_ID }}","clientSecret":"${{ secrets.CLIENT_SECRET }}","subscriptionId":"${{ secrets.SUBSCRIPTION_ID }}","tenantId":"${{ secrets.TENANT_ID }}"}'

Also one can leverage the OIDC login mechanism in this action for more secure login.


SQLAlchemy: Investigate whether and how to build upon the JSON type implementation

About
crate/crate-python#559 revealed that the SQLAlchemy implementation for supporting CrateDB's OBJECT type has drawbacks when accessing the attributes of the dictionary facade by key, i.e. indexing into the dictionary using the classical Entity.field["attribute"] notation.
Thoughts
Here, we are trying to outline a possible alternative way of implementing support for that, which may more closely align to PostgreSQL's JSON type support. While CrateDB itself does not provide support for the JSON|JSONB types, the same SQLAlchemy interface could still be leveraged to access CrateDB's OBJECT type in the same spirit. At least, this is what I am assuming for now.
Details
Please inspect SQLAlchemy's sqltypes.JSON type implementation, which is provided as a facade for vendor-specific JSON types, which are currently PostgreSQL, MySQL, SQLite, and Microsoft SQL Server. Each of them has their own specializations, so I think the chance to re-use some code from the generic type variants for implementing CrateDB's OBJECT type might actually be possible.
New interface
The new interface to index operations is based on data casters.

Index operations, i.e. those invoked by calling upon the expression using the Python bracket operator as in some_column['some key'], return an expression object whose type defaults to JSON by default, so that further JSON-oriented instructions may be called upon the result type.
However, it is likely more common that an index operation is expected to return a specific scalar element, such as a string or integer. In order to provide access to these elements in a backend-agnostic way, a series of data casters are provided. [...]
See section »Casting JSON Elements to Other Types« at the SQLAlchemy JSON type documentation.

Code examples
When adhering to the new interface, the same statement would be written slightly different.
Before
sa.select(Character).where(Character.data['x'] == 1)

After
sa.select(Character).where(Character.data['x'].as_integer() == 1)

That patch may include more improvements in this area.

GH-27


Please remove Rails/UnusedIgnoredColumns

Is your feature request related to a problem? Please describe.
This cop was added in https://github.com/rubocop/rubocop-rails/pull/494.
While the idea is good in practice it doesn't work. You should only delete the ignore_column when that column is removed from the production database, but if you have that cop enabled, it is using the development information to tell you that the column was already removed, which at that point you have no way to know yet.
This will cause the same pull request that removes the column from the database, to also remove it from the ignore_column list, negating the reason why you need to use ignore_column, and potentially causing downtime for systems.
Describe the solution you'd like
I don't see how to implement this cop is a way that would actually work since you need to have access to the production database schema to tell if a column was already removed or not, so I think this cop should be deprecated and removed.

@pocke did you find any case where you can safely know a column was already removed in production that made you implement this cop? As far as I know, checking schema.rb always will lead the cop to think the column was removed in the same pull request that just added ignore_column.


[BUG] -input flag deletes existing files

Describe the bug
I mistakenly used -input to refer to my zip file, instead of -zip, which results in the file specified by -input to be immediately deleted.
To Reproduce
(env) mbp:test irees$ ls -ldhF ok.zip
-rw-r--r--  1 irees  staff    38M Apr 15 14:32 ok.zip

(env) mbp:test irees$ java -jar gtfs-validator.jar -i ok.zip
[INFO ] 2020-04-15 14:32:53.767 [main] Main - --url and relative path to zip file(--zip option) not provided. Trying to find zip in: /Users/irees/test
[INFO ] 2020-04-15 14:32:53.785 [main] Main - zip file found: /Users/irees/test/ok.zip
[INFO ] 2020-04-15 14:32:53.786 [main] Main - --output not provided. Will place execution results in: /Users/irees/test/output
[INFO ] 2020-04-15 14:32:53.786 [main] Main - Unzipping archive
[ERROR] 2020-04-15 14:32:53.801 [main] Main - An exception occurred: /Users/irees/test/ok.zip (Is a directory)
[INFO ] 2020-04-15 14:32:53.804 [main] Main - Took 987ms

(env) mbp:test irees$ ls -ldhF ok.zip
drwxr-xr-x  2 irees  staff    64B Apr 15 14:32 ok.zip/

Expected behavior
Fail when -input path exists.
Witnessed behavior
See above
Screenshots
See above
Environment used

OS: macOS 10.14.4
Java version: openjdk 14 2020-03-17

A couple other notes: 1. it runs the delete before checking if the -zip path is present or valid gtfs  2. it's a recursive delete of the entire tree.

Thanks for the insight @irees, MobilityData will be exploring this.

Hey @irees ,
-input path will exist if the validator has been run previously. Not sure we should fail in that case?

@fabrice-v I would suggest considering re-naming the -input to something different, as I think it's confusing to users. I think most end users would view what it does (serve as unzipped location of a GTFS zip file) as a cache or working directory, not as input to the application. The user-facing "input" is really the -zip param.

That is a good point indeed. Shall we keep the possibility to specify where said cache is created which was the original intent or simplify it to something like 'an input folder will be created with the content of the provided dataset?

@fabrice-v I think it's a good idea to keep the cache location configurable via command line, as these files can get huge and users may not want to unzip to the same drive as where they are running the application.

Alright here is how I propose to move forward

rename -zip to -inputzip
rename -input to -extract
Thoughts?

SGTM!

Alright here is how I propose to move forward

rename -zip to -inputzip
rename -input to -extract
Thoughts?

This had been implemented in d55683b78aa905bf0b8ad186ae66a4483898928a

@lionel-nj Could you update the release docs in that PR too? https://github.com/MobilityData/gtfs-validator/blob/master/RELEASE.md

Sure, working on it.

Note that in the final version of the fix -zip was renamed to -zipinput


Switch to Front camera but Failed.

Stackoverflow Post here
I just change 2 Lines of the project to switch to the front camera but it failed.  Under the path it generate a MP4 file, but my cell phone failed to open it.  (real device debugging)

By changing mMediaRecorder.setProfile(CamcorderProfile.get(1,CamcorderProfile.QUALITY_HIGH)); ==> mMediaRecorder.setProfile(CamcorderProfile.get(1,CamcorderProfile.QUALITY_LOW));
it works well, but I don't know why?

I think your front camera doesn't support the highest quality. You can use CamcorderProfile.hasProfile(int cameraId, int quality) to check if the quality is supported by specified camera. See http://developer.android.com/reference/android/media/CamcorderProfile.html


Engine.CustomCommand() is not really a custom command

Suggestion for improvement:

As an user I want to be able to formalise a full FFMpeg command and expect the method to accept everything as it is without adding any additional parameter or conversion.

2nd suggestion:

As an user I would expect the Engine.CustomCommand() to be executed right away or to be added to the task queue for later execution.

It all started when I wanted to cut and merge my video without re-encoding. I have the following commands what I know work well:

ffmpeg -ss 30 -t 60 -i D:\VideoCutting\Sample.avi -c copy -avoid_negative_ts make_zero -fflags +genpts D:\VideoCutting\Segment1.ts
ffmpeg -ss 3600 -t 60 -i D:\VideoCutting\Sample.avi -c copy -avoid_negative_ts make_zero -fflags +genpts D:\VideoCutting\Segment2.ts
ffmpeg -i "concat:D:\VideoCutting\Segment1.ts|D:\VideoCutting\Segment2.ts" -c copy -flags +global_header -fflags +genpts D:\VideoCutting\Merged.mp4

Converted them into two separate CustomCommand. I learned the following:

I had to remote the 'ffmpeg' at the start. Fine with that.
The CustomCommand is not executed right away.
Had to call the Engine.Convert() method.
The Engine.Convert() method did something which took so long time I had to kill the FFMPeg process from the Task Manager.
The Engine.CustomCommand() and Engine.Convert() did something to my commands which caused the video to be re-encoded. I was NOT able to execute my commands as they were to avoid the fr34king re-encoding the 6GB file!
I was not able to batch the custom commands for "conversion".
I was not able to cancel currently running task.
The Engine.Dispose() did not end or kill the FFMpeg background task!

Did I misunderstood or do something wrong? I apologise if I did.

The issue that you encountered was because you called Engine.Convert, CustomCommand doesn't raise any progress or completion events, so please make sure that Engine.CustomCommand really is doing nothing and I can investigate further.

You are right, my mistake. The engine.CustomCommand(firstPartCommand) produces a video file just fine. Just any following custom command is not executed:
engine.CustomCommand(firstPartCommand); // executes just fine
engine.CustomCommand(secondPartCommand); // not executed and no progress or completion event is rised!

Probably I should dispose the current Engine and use a fresh instance of the Engine. Is it indended to use it so?
However, the CustomCommand DOES rise a completion and should also rise the progress event if the job would be not that small as mines was.
The CustomCommand and Convert call the same StartFFmpegProcess method:
// public void CustomCommand(string ffmpegCommand)
this.StartFFmpegProcess(new EngineParameters()
{
      CustomArguments = ffmpegCommand
});

vs
// private void FFmpegEngine(EngineParameters engineParameters)
try
{
      this.Mutex.WaitOne();
      this.StartFFmpegProcess(engineParameters);
}
finally
{
      this.Mutex.ReleaseMutex();
}

There's definitely a bug there, not being able to use the same instance of Engine after the first conversion...
I'm refactoring a lot of the things in there at the moment and unit testing it all as I go along. Instead of attempting to serialize the raw data that is put out byFFmpeg, I'll be raising new events to push that data out to the client. I think that's the most sensible solution as we'll never be able to cover every use case.


Feature: Connection Time Out feature update

PR Description
this PR is not complete creted for a review
Related Issues

Related Issue #312
Closes #312

Checklist

[x] I have gone through the contributing guide
[x] I have run the tests (flutter test) and all tests are passing

Added/updated tests?
We encourage you to add relevant test cases.

[ ] Yes
[x] No, and this is why: its a feature

@animator sir can you please review this as it is showing the time out error but the thing is once the response is recieved from thr API side the timeout error is replaced by the actual response please guide me to tackle this problem i have used flag but the code is changing the its value

@animator sir the feature is successfully implemented please have a look

@nikkivirtuoso Please cleanup the code. The quality of code you currently submitted is not acceptable. Do some research on how you can write clean codes.

sorry sir @animator i forgot to remove the commented code and did some spelling mistakes now i have removed the unnecessary things please have a look

@animator @ashitaprasad please have a look i have cleaned the code

@animator i have stored the value of connectiontimeout in a timeout variable if still my code is unacceptable please guide me so that I can get to know which part i need to make changes

@nikkivirtuoso I have added pointers to places where you need to think more on your approach.

@animator , so you are saying that i don't have to make changes in the existing file, i have to make another file and do all the stuff right ? something like wrapping.

No. I am saying the changes pointed above are incorrect.


ConnectAttempt fixed

I am closing this PR as the request merges release into develop, and all commits included are already in develop. @erl919, if there's something you're trying to achieve, please let us know. I'd be happy to help.


Generate initialization from seasonal average file.

As suggested by @sdc50, a smaller file to have on the compute server would be the seasonal initialization file with 365 days of seasonal averages from 35 years of historical data.

Also, it would be good to have a way to display this as a chart to the user.


feedback - Removed usages and references of @backstage/backend-common

Hey, I just made a Pull Request!
Removed usages and references of @backstage/backend-common
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

I've made some corrections but I'll need your feedback as to how best to resolve the API Report errors in the CI. It want me to export the RouterOptions but ideally we should be removing that and the export of createRouter and pushing Adopters to using the New Backend System.

Reason for closing outlined here: https://github.com/backstage/community-plugins/issues/1176#issuecomment-2489256602


Request support for site www.cinetecamilano.it

Checklist

[x ] I'm reporting a new site support request
[x ] I've verified that I'm running youtube-dl version 2020.09.20
[x ] I've checked that all provided URLs are alive and playable in a browser
[x ] I've checked that none of provided URLs violate any copyrights
[x ] I've searched the bugtracker for similar site support requests including closed ones

Example URLs

Single video: https://www.cinetecamilano.it/film/1784
Description

WRITE DESCRIPTION HERE
Credentials for the site can be obtained by registering.
Here's the dump.
$ youtube-dl --verbose https://www.cinetecamilano.it/film/1784
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://www.cinetecamilano.it/film/1784']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2020.09.20
[debug] Python version 3.7.2 (CPython) - Windows-10-10.0.16299-SP0
[debug] exe versions: ffmpeg N-94609-gc87237d105, ffprobe N-94609-gc87237d105
[debug] Proxy map: {}
[generic] 1784: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 1784: Downloading webpage
[generic] 1784: Extracting information
ERROR: Unsupported URL: https://www.cinetecamilano.it/film/1784
Traceback (most recent call last):
File "c:\personal\python\python37-32\lib\site-packages\youtube_dl\YoutubeDL.py", line 797, in extract_info
ie_result = ie.extract(url)
File "c:\personal\python\python37-32\lib\site-packages\youtube_dl\extractor\common.py", line 532, in extract
ie_result = self._real_extract(url)
File "c:\personal\python\python37-32\lib\site-packages\youtube_dl\extractor\generic.py", line 3382, in _real_extract
raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.cinetecamilano.it/film/1784

Hi.
Is the issue still open because I missed or provided some wrong information?


Add fluka.cern data entry in table

Add entry in other-results-table for the data produced for the geant4 to fluka comparison.

Accepting PR #47.


Add tutorial: GC MS data processing [metabolomics]

This PR adds a new tutorial to the metabolomics category. The tutorial is focusing on explanation of GC-MS data processing using tools XCMS, RAMClustR, RIAssigner, and matchms.

Pinging @melpetera @lecorguille @foellmelanie if any of you could look at it as metabolomics maintainers :)

Hi,
Thank you for providing this new training material.
If by next wednesday nobody have done it yet, I will try to have a look at the content from a "metabolomic user" point of view (although I am no expert in GC-MS). Sorry for the delay.

Hi there,
I just finished to try this tutorial. Great job there :)
I have a few comments and suggestions. I am not used to reviewing on GitHub, so I will try to use the dedicated functionnalities, but I appology in advance in case I do not use the system correctly.
I will do this tomorrow morning, sorry for the delay.

Thank you for this new tutorial! Reviewing this was the opportunity to test the corresponding tools and get an overview of the outputs :) I made some suggestions this morning, I hope I did it the right way (it seems to be ok, at least it looks similar to the reviews I have seen on GitHub!).

@melpetera Thank you for a very detailed review! I tried to cover all the issues you discovered. Let me know if more needs to be done.

@hexylena and @melpetera thank you very much for the review and spending the time on improving the training! Your feedback is highly appreciated :).

I know it may seems that I split hairs here, but to note this is because I think this tutorial is great, so I can't help suggesting what I think could make it even better :flushed:

I know it may seems that I split hairs here, but to note this is because I think this tutorial is great, so I can't help suggesting what I think could make it even better flushed

@melpetera I tried to address your last comments. We are very grateful for your feedback and suggestions, they are very valuable!

@melpetera we added the already processed XCMS dataset to the Zenodo record and added a section stating that this can be used to skip the pre-processing part - like this we can keep things open if we want to add some specific about GC peak picking but also users have the option to start from the other tutorial or skip this section entirely.
The version for RIAssigner has been updated, so I think this should be all good now :).

Thank you for your work and the time spent on my suggestions!
All good for me :)
@hexylena Am I supposed to hit the "workflow" button now?

done so now. looks good! if it passes we merge :)

@melpetera and @hexylena thank you so much for the support! :)


README: refactor a bit

Make the first section more punchy. Add Contributing and License section
for good measure.

Thanks a lot. This looks better.


Husky testing

When dialing a barcode, deep link to the PCDEQC form where applicable.

 Update 'duplicate_record_ids' table in the Switchboard

 Add Husky Testing REDCap to SCAN Switchboard

 Rename internal mentions of "SCAN switchboard"

I just pushed a fixup to reverse the logic in the latest "Fix" commit so the conditional that resolves to True comes before the one that resolves to False.

I force-pushed just to move the duplicate_record_ids commit further down in the list. All new changes are in new "fixup!" or "squash!" commits.

Rebased onto master.


Update an app icon in README

What's this PR do?

[ ] Add new routes for user
[ ] Add instruction for using docker-compose

What are the relevant Git tickets?
Screenshots (if appropriate)

Any background context you want to provide? (if appropriate)

Is there a blog post?
Does the knowledge base need an update?
Does this add new dependencies which need to be added to?

Thanks for the update


Negative offsets are ignored in layoutData on Android

Negative offsets in layout data are ignored, i.e. treated as zero.
Nexus 5x, Android 7.1.2
new tabris.Composite({
  id: 'first',
  left: 20, top: 20, width: 100, height: 100,
  background: 'red'
}).appendTo(tabris.ui.contentView);

new tabris.Composite({
  left: ['#first', -50], top: ['#first', -50], width: 100, height: 100,
  background: 'green'
}).appendTo(tabris.ui.contentView);

The missing supported for negative margins in the android layout library is tracked at https://issuetracker.google.com/issues/37136609

If we can not fix this we should at least align the behavior of iOS and Android.

There is no foreseeable support for the negative margins on Android. To align the behavior we should prohibit the negativ margin on all platforms.

Didn't mean to close the issue.

Follow-up issue: #1727


Option to configure SSL/TLS verification level

Feature Request
Need an option to only skip host name verification of the server certificate.
Typically when using AWS Redis the host name is a random name.
So we use a pre-defined DNS name for it instead but then the certificate by AWS will not match the hostname.
There is an option called "verifyPeer" but when that is set to false not only host name verification is skipped also all kinds of verification of certificates are skipped. So that option can't be use in production.
Describe the solution you'd like
A dedicated option to only skip host name verification just like standard SSL factory has.
Describe alternatives you've considered
Teachability, Documentation, Adoption, Migration Strategy

What a few drivers provide is a SSL mode enum (like disabled, enabled, verify ca, verify full). On the other side, there are several ways how to verify a certificate (subject/SAN check, issuer trust relationship, validity period, fingerprinting). Especially fingerprint checking gains on popularity. I wonder whether we can combine these aspects somehow.

Probably, there's a misconception. Java's HostnameVerifier runs after TrustManager has performed the SSL handshake and has validated certificates. Note that you can already configure a custom TrustManagerFactory.
What we can do here is introducing an SSL verification level as we're additionally configuring SSLParameters.setEndpointIdentificationAlgorithm("HTTPS") by default.
That intent could be expressed through RedisUri.setVerifyPeer(Verification.CERT) vs. RedisUri.setVerifyPeer(Verification.FULL) (corresponds with setVerifyPeer(true)).

Is there any other option to implement this than providing custom X509ExtendedTrustManager to sslContextBuilder in SslConnectionBuilder  ?
sslContextBuilder.trustManager(new TrustManagerFactory(new TrustManagerFactorySpi() {
    @Override
    protected void engineInit(KeyStore keyStore) {
        // NOOP
    }
    @Override
    protected TrustManager[] engineGetTrustManagers() {
        // Provide a custom trust manager, this manager trust all certificates
        return new TrustManager[] {
                new X509ExtendedTrustManager() {
                    @Override
                    public void checkClientTrusted(final X509Certificate[] chain, final String authType,
                            final Socket socket) throws CertificateException
                    {
                        // NOOP
                    }

                    @Override
                    public void checkServerTrusted(final X509Certificate[] chain, final String authType,
                            final Socket socket) throws CertificateException
                    {
                        // NOOP
                    }

                    @Override
                    public void checkClientTrusted(final X509Certificate[] chain, final String authType,
                            final SSLEngine engine) throws CertificateException
                    {
                        // NOOP
                    }

                    @Override
                    public void checkServerTrusted(final X509Certificate[] chain, final String authType,
                            final SSLEngine engine) throws CertificateException
                    {
                        // NOOP
                    }

                    @Override
                    public void checkClientTrusted(
                            java.security.cert.X509Certificate[] x509Certificates, String s) {
                        // NOOP
                    }

                    @Override
                    public void checkServerTrusted(
                            java.security.cert.X509Certificate[] x509Certificates, String s) {
                        // NOOP
                    }

                    @Override
                    public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                        return EmptyArrays.EMPTY_X509_CERTIFICATES;
                    }
                }
        };
    }

    @Override
    protected void engineInit(ManagerFactoryParameters managerFactoryParameters) {
    }
}, null, TrustManagerFactory.getDefaultAlgorithm()) {
});

Is there any other option to implement this than providing custom X509ExtendedTrustManager to sslContextBuilder in SslConnectionBuilder  ?
sslContextBuilder.trustManager(new TrustManagerFactory(new TrustManagerFactorySpi() {
    @Override
    protected void engineInit(KeyStore keyStore) {
        // NOOP
    }
    @Override
    protected TrustManager[] engineGetTrustManagers() {
        // Provide a custom trust manager, this manager trust all certificates
        return new TrustManager[] {
                new X509ExtendedTrustManager() {
                    @Override
                    public void checkClientTrusted(final X509Certificate[] chain, final String authType,
                            final Socket socket) throws CertificateException
                    {
                        // NOOP
                    }

                    @Override
                    public void checkServerTrusted(final X509Certificate[] chain, final String authType,
                            final Socket socket) throws CertificateException
                    {
                        // NOOP
                    }

                    @Override
                    public void checkClientTrusted(final X509Certificate[] chain, final String authType,
                            final SSLEngine engine) throws CertificateException
                    {
                        // NOOP
                    }

                    @Override
                    public void checkServerTrusted(final X509Certificate[] chain, final String authType,
                            final SSLEngine engine) throws CertificateException
                    {
                        // NOOP
                    }

                    @Override
                    public void checkClientTrusted(
                            java.security.cert.X509Certificate[] x509Certificates, String s) {
                        // NOOP
                    }

                    @Override
                    public void checkServerTrusted(
                            java.security.cert.X509Certificate[] x509Certificates, String s) {
                        // NOOP
                    }

                    @Override
                    public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                        return EmptyArrays.EMPTY_X509_CERTIFICATES;
                    }
                }
        };
    }

    @Override
    protected void engineInit(ManagerFactoryParameters managerFactoryParameters) {
    }
}, null, TrustManagerFactory.getDefaultAlgorithm()) {
});

That's in place now via RedisURI.setVerifyPeer(CA).

That's in place now via RedisURI.setVerifyPeer(CA).

can TLS 1.2 be configured now using Lettuce??

Yes, via SslOptions.sslParameters(…).

cool thanks

setVerifyPeer

If we are using Spring Data Redis with Lettuce Client then how can we set this flag. Currently I do not see any provision to set this up who are using spring data redis