Reconnection leads to loss of WorkingDirectory

FTP OS: Windows
FTP Server: WindowsServerIIS
Computer OS: Windows 10 20H2 (Build 19042.1526)
FluentFTP Version: 37.0.1.
Bug:
In the end, I run the following code (this is copied together a bit to keep it short):
var _client = new FtpClient("<IP_ADDRESS>", "21341", @"DOMAIN\username", "******");
_client?.Connect();

var targetDir = "/BLOB/03 MyData/03 DataChanges";

if (!_client.DirectoryExists(targetDir))
    _client.CreateDirectory(targetDir);
_client.SetWorkingDirectory(targetDir);

foreach(var ftpFile in _client.GetListing())
{
    var destFile = Path.Combine(@"D:\TOOL\LOGS\MyData\DataChanges\", RemotePaths.GetFtpFileName(ftpFile.FullName));
    await _client.DownloadFileAsync(ftpFile.Name, destFile, FtpLocalExists.Overwrite);

    // Doing things with the file that may take some time

    _client.MoveFile(ftpFile.Name, $"./Archiv/{RemotePaths.GetFtpFileName(ftpFile.FullName)}", FtpRemoteExists.Overwrite);
}

This code also works perfectly. But in rare cases moving the file to the archive does not work. I have tried to trace this with the log.
It seems that the client sometimes loses the connection, and then reestablishes it. However, it loses the information of the current WorkingDirectory which causes the move not to work.
Logs :

# Connect()
Status:   Connecting to <IP_ADDRESS>:21341
Response: 220 Microsoft FTP Service
Status:   Detected FTP server: WindowsServerIIS
Command:  USER DOMAIN\username
Response: 331 Password required
Command:  PASS ***
Response: 230 User logged in.
Command:  FEAT
Response: 211-Extended features supported:
Response: LANG EN*
Response: UTF8
Response: AUTH TLS;TLS-C;SSL;TLS-P;
Response: PBSZ
Response: PROT C;P;
Response: CCC
Response: HOST
Response: SIZE
Response: MDTM
Response: REST STREAM
Response: 211 END
Status:   Text encoding: System.Text.UTF8Encoding
Command:  OPTS UTF8 ON
Response: 200 OPTS UTF8 command successful - UTF8 encoding now ON.
Command:  SYST
Response: 215 Windows_NT
Status:   Listing parser set to: Windows

# DirectoryExists("BLOB/03 MyData/03 DataChanges")
Command:  PWD
Response: 257 "/" is current directory.
Command:  CWD BLOB/03 MyData/03 DataChanges
Response: 250 CWD command successful.
Command:  CWD /
Response: 250 CWD command successful.

# SetWorkingDirectory("BLOB/03 MyData/03 DataChanges")
Command:  CWD BLOB/03 MyData/03 DataChanges
Response: 250 CWD command successful.

# GetListing(null, Auto)
Command:  PWD
Response: 257 "/BLOB/03 MyData/03 DataChanges" is current directory.
Command:  TYPE I
Response: 200 Type set to I.

# OpenPassiveDataStream(AutoPassive, "LIST /BLOB/03 MyData/03 DataChanges", 0)
Command:  EPSV
Response: 229 Entering Extended Passive Mode (|||21398|)
Status:   Connecting to <IP_ADDRESS>:21398
Command:  LIST /BLOB/03 MyData/03 DataChanges
Response: 125 Data connection already open; Transfer starting.
+---------------------------------------+
Listing:  02-14-22  04:54PM       <DIR>          Archiv
Listing:  02-11-22  09:36AM       <DIR>          Error
Listing:  02-21-22  12:05PM                98621 TestFile-MyData-Export-21-02-2022.xlsx
-----------------------------------------
Status:   Disposing FtpSocketStream...

# CloseDataStream()
Response: 226 Transfer complete.
Status:   Disposing FtpSocketStream...
Status:   Confirmed format Windows

# DownloadFileAsync("D:\TOOL\LOGS\MyData\DataChanges\TestFile-MyData-Export-21-02-2022.xlsx", "TestFile-MyData-Export-21-02-2022.xlsx", Overwrite, None)

# OpenReadAsync("TestFile-MyData-Export-21-02-2022.xlsx", Binary, 0, 0)

# GetFileSizeAsync("TestFile-MyData-Export-21-02-2022.xlsx", -1)
Command:  SIZE TestFile-MyData-Export-21-02-2022.xlsx
Response: 213 98621

# OpenPassiveDataStreamAsync(AutoPassive, "RETR TestFile-MyData-Export-21-02-2022.xlsx", 0)
Command:  EPSV
Response: 229 Entering Extended Passive Mode (|||21351|)
Status:   Connecting to <IP_ADDRESS>:21351
Command:  RETR TestFile-MyData-Export-21-02-2022.xlsx
Response: 125 Data connection already open; Transfer starting.
Status:   Disposing FtpSocketStream...
Response: 226 Transfer complete.

# MoveFile("TestFile-MyData-Export-21-02-2022.xlsx", "./Archiv/TestFile-MyData-Export-21-02-2022.xlsx", Overwrite)

# FileExists("TestFile-MyData-Export-21-02-2022.xlsx")
Status:   Testing connectivity using Socket.Poll()...
Status:   Disposing FtpSocketStream...

# Connect()
Status:   Connecting to <IP_ADDRESS>:21341
Response: 220 Microsoft FTP Service
Status:   Detected FTP server: WindowsServerIIS
Command:  USER DOMAIN\username
Response: 331 Password required
Command:  PASS ***
Response: 230 User logged in.
Command:  FEAT
Response: 211-Extended features supported:
Response: LANG EN*
Response: UTF8
Response: AUTH TLS;TLS-C;SSL;TLS-P;
Response: PBSZ
Response: PROT C;P;
Response: CCC
Response: HOST
Response: SIZE
Response: MDTM
Response: REST STREAM
Response: 211 END
Status:   Text encoding: System.Text.UTF8Encoding
Command:  OPTS UTF8 ON
Response: 200 OPTS UTF8 command successful - UTF8 encoding now ON.
Command:  SYST
Response: 215 Windows_NT
Status:   Listing parser set to: Windows
Command:  SIZE /BLOB/03 MyData/03 DataChanges/TestFile-MyData-Export-21-02-2022.xlsx
Response: 213 98621

# FileExists("./Archiv/TestFile-MyData-Export-21-02-2022.xlsx")
Command:  PWD
Response: 257 "/" is current directory.
Command:  SIZE /Archiv/TestFile-MyData-Export-21-02-2022.xlsx
Response: 550 The system cannot find the path specified.

# Rename("TestFile-MyData-Export-21-02-2022.xlsx", "./Archiv/TestFile-MyData-Export-21-02-2022.xlsx")
Command:  RNFR /TestFile-MyData-Export-21-02-2022.xlsx
Response: 550 The system cannot find the file specified.
Command:  QUIT
Response: 221 Goodbye.
Status:   Disposing FtpSocketStream...

# Dispose()
Status:   Disposing FtpClient object...
Status:   Disposing FtpSocketStream...

Yes working directory is not restored.

Added to the bucket list. We will pick this up as and when we have free time. Comment on this issue if you want us to prioritize it. Thanks!

This has been done. Removing from the bucket list.


Normalize pv and gsp coords

Detailed Description
Normalize the pv and gsp, x and y coordinates.
Need to find the max and min (x and y) values of these from all batches.
Context
Useful to normalize, so that these values can be included in ML models

Might be good to normalize them by the same amount, so its easier for ML model to align them

Both x and y minimums can be 0
Max for both PV and GSP is
x_max = 65665
y_max = 1151577
note we get rid of anything above y>1_036_975 - https://github.com/openclimatefix/Satip/issues/30


calculation of 2023 spring equinox is off by over 1:30

Thank you for this great library.  It's been super helpful.
I'm calculating equinoxes and solstices with SwiftAA and noticing over a minute and a half difference from various sources on the internet.  For example, most sources say it will be at 21:24, but SwiftAA calculates it as 21:25:34.
Is this degree of inaccuracy normal for this library?
I'm curious if this is due to being out of date with the AA+ library which is up to v2.43 in 2022, whereas we're still using a the 2.0.9 version from 2019?  Would it be much work to update AA+ to the newest version?  I noticed in the change logs that various renames happened.  Does this affect ObjCAA?

After doing some more research I found that the underlying AA+ library has a comprehensive test suite and I discovered that to get an accurate UTC value, we need to use the TT2UTC function.  This raises questions of the very convenient JulianDay(_ date:Date) constructor.  Does this convert from UTC2TT before returning the JD value?  When calling JulianDay(Date()).date does it convert from TT2UTC?  Should it?

Thanks for your message @thosmos No, 1.5 minute off is not a normal accuracy for this library!
The constructor of the JulianDay from the Date isn't doing any conversion so far, but it seems we might need to reconsider the question.
The update of to a new version of AA+ library isn't too much work. Replacing the files, running the tests, correcting the calls that may have change. Would you mind making a PR?


feat: respect helmfile.lock with ad-hoc kustimzation

Change ensures that dependencies are resolved in state.go prior to creating releases.
That way releases has proper version when helmfile.lock is present.

@vlpav030 where is the issue link?

resolves #1229
@yxxhero I've also edited PR title to make it more clear.

@vlpav030 Adding more tests will be better.

@vlpav030 thanks so much. LGTM


ci: enable eth_getProof hive test

From these PR's being complete
https://github.com/paradigmxyz/reth/pull/5071
https://github.com/paradigmxyz/reth/pull/2067
And this issue being resolved
https://github.com/paradigmxyz/reth/issues/2046
I'm assuming eth_getProof hive tests can be enabled now?

@KolbyML do eth_getProof tests also query for historical proofs?

Didn't check, I will look in the morning

It doesn't do historical (for now at least). There are three tests:

One that fetches with a block hash which requests for the block with hash 0x8f64c4436f7213cfdf02cfb9f45d012f1774dfb329b8803de5e7479b11586902
One that fetches for latest
One that fetches for latest with storage

The tests: https://github.com/ethereum/execution-apis/blob/main/tests/eth_getProof
The chain.rlp is generated using https://github.com/lightclient/rpctestgen and the FCU matches 0x8f64c4436f7213cfdf02cfb9f45d012f1774dfb329b8803de5e7479b11586902: https://github.com/lightclient/rpctestgen/blob/main/chain/headfcu.json


[WA-26] Add user selection window in installer

What does this PR do?

Adds a dialog to the Windows MSI installer that allows the user to input the agent username and/or password.

The input fields are linked to the DDAGENTUSER_NAME and DDAGENTUSER_PASSWORD properties and will be pre-filled if these options are supplied on the command line.
Upon clicking "next" a custom action will validate the input, and display an error dialog if an error is detected.

Motivation

customer requests
Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes
Supply DDAGENTUSER_NAME and DDAGENTUSER_PASSWORD on the command line, check input fields are pre-filled
Check each of the cases from canInstall() and verify the corresponding errorMessage appears after clicking the "Next" button

Reviewer's Checklist

[ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
[ ] A release note has been added or the changelog/no-changelog label has been applied.
[ ] Changed code has automated tests for its functionality.
[ ] Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
[ ] At least one team/.. label has been applied, indicating the team(s) that should QA this change.
[ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
[ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
[ ] If applicable, the k8s/<min-version> label, indicating the lowest Kubernetes version compatible with this feature.
[ ] If applicable, the config template has been updated.

Can you add new behavior picture to contrast with old (which is great)

no docs review required
@cswatt the PR contains user facing Installation UI and Installation error messages. Is anyone in documentation should or can review it?

@julien-lebot and @clarkb7 what was original point with catch(...)? If you switch to std:::exception you will not catch many cases. Also, depending on how we build DLL we may miss structured exceptions on top of it.

@julien-lebot and @clarkb7 what was original point with catch(...)? If you switch to std:::exception you will not catch many cases. Also, depending on how we build DLL we may miss structured exceptions on top of it.

This part of the code is not meant to catch structured exceptions. catch(...) doesn't give the exception that was thrown, it simply hides it. We should stick to standard C++ everywhere as much as possible, and resort to non-standard code only when it's not possible to do otherwise.

Ah, Ok, in this function I guess it is sufficient since we call std lib code (although it will not catch something like throw E_FAIL.


Improvement of navbar responsive and design on phone

Please assign me the task for improving it under gssoc extended and hacktoberfest

Ok


overflow-x-scroll has an !important applied negating respinsive directives

Prerequisites

[X] I have searched for duplicate or closed issues
[X] I have validated any HTML to avoid common problems
[X] I have read the contributing guidelines

Describe the issue
.overflow-x-scroll class has !important directive applied to it. This seems like a bug, since it overrides respoisive classes like .overflow-x-md-auto
Result from DevTools
_utilities.scss:67
.overflow-x-scroll {
    overflow-x: scroll !important;
}

but
_utility.scss:13
.overflow-x-scroll {
    overflow-x: scroll;
}

and
_utility.scss:28
.overflow-x-md-auto {
    overflow-x: auto;
}

Reduced test cases
https://codepen.io/olia-bn/pen/vYMVppd
What operating system(s) are you seeing the problem on?
Windows
What browser(s) are you seeing the problem on?
Chrome
What version of Bootstrap are you using?
5.3.3

Im a teapot.


Error after installing / ImportError: libstreamhtmlparser.so.0

Hi, thanks for great web proxy script. I wanted to give it a try, but i couldn't. I downloaded and tried to install as mentioned in documents. I installed requirements. I also built and installed streamhtmlparser in the include directory. These steps is passed without any error. But at last, when I tried to run, I got following error:
Traceback (most recent call last):
  File "Proxy.py", line 38, in <module>
    from Page import Page
  File "/opt/SwiperProxy/swiperproxy/Page.py", line 23, in <module>
    from streamhtmlparser import *
ImportError: libstreamhtmlparser.so.0: cannot open shared object file: No such file or directory

I should say, I am a little bit new at python. So I may miss something obvious. If so, sorry for taking time. Thanks.

same issue

Sorry, the project is not really maintained anymore and I am currently not at home. I will try to compile it when I am back from my holiday and see what the issue is. Something probably changed with libstreamhtmlparser, a workaround might be to pick a version from a year ago?

can you please tell maybe version of it

tried v1.0 dont work

I don't know yet, sorry. I will try it out somewhere this week

Copy the Python packages to a location where Debian will recognize them:
$ cp /usr/local/lib/python2.7/dist-packages/* /usr/lib/python2.7/dist-packages
$ ldconfig -v
Did you try that? What python version do you run? 2 or 3?

Iam using python2 and i have tried that and that did not fix it.

Cant we make like a pip package for this library atleast for debian based systems

Any updates? Even I am facing the same issue.

Same problem here...

Same here


WSL2 mounted host drive access hangs after a short while

Version
Microsoft Windows [Version 10.0.22621.1265]
WSL Version

[X] WSL 2
[ ] WSL 1

Kernel Version
<IP_ADDRESS>-microsoft-standard-WSL2
Distro Version
docker-desktop
Other Software
Docker Desktop
Repro Steps
Use Docker Desktop to build some software using ninja -j9 in an Ubuntu 20.04 LTS container.
Expected Behavior
The build runs fast.
Actual Behavior
The build gets slower and slower until it eventually hangs.
Once it hangs, running df -ah in the WSL2 instance hangs right before it shows the /mnt point of the drive.
wsl --shutdown also hangs.
Diagnostic Logs
No response

Noticing continued high reported CPU usage in Task Manager after running a docker container in WSL2 for a while...

Except this CPU usage is not inside the VM. And at the same time Task Manager also doesn't report any actual process using this much CPU.

Yet the CPU usage graphs remain high... Doing nothing?
No difference after wsl --shutdown

Also of note, while the build processes are running inside the WSL2, Task Manager in the Details and Processes pages runs extremely sluggish.

(Using Docker Desktop with Hyper-V backend instead now, it goes super fast.)

Your CMD window title is wsl -d docker-desktop. The Docker distros  docker-desktop and docker-desktop-data are not to be invoked directly.

Hi, it's just to check if the processes in there are doing anything, and to find out what is blocking.
On Sat, Feb 18, 2023, at 23:35, zcobol wrote:

Your CMD window title is wsl -d docker-desktop. The Docker distros docker-desktop and docker-desktop-data are not to be invoked directly.
—
Reply to this email directly, view it on GitHub https://github.com/microsoft/WSL/issues/9654#issuecomment-1435701401, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMB77NZVOZKAIHKJ3OGUF3WYDT4DANCNFSM6AAAAAAVAJTOYU.
You are receiving this because you authored the thread.Message ID: @.***>

I also noticed heavy access to the ext4 drive image after a while (after the mounted host drive starts becoming slow and getting stuck). Seems to be some heavy swapping going on once that happens?
(I already tried upping the WSL2 memory to 24GB, host memory is 64GB, still the same behavior.)

+1 - I also experience this same issue with Docker for Desktop under WSL2.

Same issue after sometime with heavy read, WSL2 mounted host disk access hangs !!! HELP!

Same issue under WSL2.
WSL2 mounted host disk access hangs in 20 minutes,and I execute wsl --shutdown...

+1
I also have the same issue, the disk being mounted is a windows storage space device.

The same issue, help

Same here!!

Same issue !!

same when running node.js building or linting


Implement improved code examples in storybook

In cases that need functions in storybook examples, we should create a demo component file and reference the code like here:

List of examples we should change:

https://storybook.onyx.schwarz/?path=/docs/feedback-dialog--docs -> modal
https://storybook.onyx.schwarz/?path=/docs/form-select--docs -> withCustomSearch
... TODO comb through alle examples

Full example context


rake attachinary:install:migrations -> Index table names are too long

After running rake attachinary:install:migrations I get the following output
WARNING: Use strings for Figaro configuration. 587 was converted to "587".
==<PHONE_NUMBER>5142 CreateAttachinaryTables: migrating ==========================
-- create_table(:attachinary_files)
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

Index name 'index_attachinary_files_on_attachinariable_type_and_attachinariable_id' on table 'attachinary_files' is too long; the limit is 63 characters

The fix is to add an own index name to the attachinariable reference. t.references :attachinariable, polymorphic: true, index: {name: "index_attachinary_files_attachinariable"}. Also note that I added [5.1] to the ActiveRecord::Migration Class because it was missing. (I am running Rails 5.1 so add your version)
class CreateAttachinaryTables < ActiveRecord::Migration[5.1]
  def change
    create_table :attachinary_files do |t|
      t.references :attachinariable, polymorphic: true, index: {name: "index_attachinary_files_attachinariable"}
      t.string :scope

      t.string :public_id
      t.string :version
      t.integer :width
      t.integer :height
      t.string :format
      t.string :resource_type
      t.timestamps
    end
    add_index :attachinary_files, [:attachinariable_type, :attachinariable_id, :scope], name: 'by_scoped_parent'
  end
end

Thanks @mklb


Implement python binding

must be a separate python module that anyone can use
for now, only make a binding for vdp usb gadget api
make some test python script that'll demonstrate that the whole thing works

Done, we now have bindings both for low-level usb api and gadget api


feat(api): Accepting all types for attributes values

Note: https://github.com/optimizely/ruby-sdk/blob/0e9980a39e81a19cb368c2188b135d7450551516/lib/optimizely/helpers/validator.rb#L43
ATTRIBUTE_VALID_TYPES.include?(attribute_value.class) can't be used as Fixnum and Bignum are deprecated in ruby 2.4 and Rubocop fails: Lint/UnifiedInteger: Use Integer instead of Fixnum.

Coverage remained the same at 100.0% when pulling 0e9980a39e81a19cb368c2188b135d7450551516 on rashid/attribute-types into e91cd75ea70f7b158941ca12d6d39a89bc66ceed on master.

Going to wait on this until we resolve and merge the Python PR, then we should make sure this matches the requirements over there as well @msohailhussain

build

@rashidsp looks like the test failures are legit. There are some lint errors.


DOTNET_NUGET_SIGNATURE_VERIFICATION

Type of issue
Other (describe below)
Description

LINUX
Important
Although signed-package verification functionality was added in .NET 5 SDK's, the functionality isn't supported on Linux until .NET 6.0.400 SDK. Don't use signed-package verification with .NET SDK versions earlier than 6.0.400.
Prior to .NET 8 SDK, verification is disabled by default during package restore operations. To opt in, set the environment variable DOTNET_NUGET_SIGNATURE_VERIFICATION to true.

After a clean install on Arch Linux via the install.sh script, I proceeded to install the maui-android nuGet package with the following command:
dotnet workload install android-maui

the NuGet package signature verification is skipped (see output.txt)
[output.txt](https://github.com/dotnet/docs/files/15326820/output.txt
As quoted above the documentation says that prior to .NET 8 sdk verification is disabled by default, so I assume that the verification on the version 8.0.204 should be enabled by default.
Page URL
https://learn.microsoft.com/en-us/dotnet/core/tools/nuget-signed-package-verification
Content source URL
https://github.com/dotnet/docs/blob/main/docs/core/tools/nuget-signed-package-verification.md
Document Version Independent Id
ccd7b834-2f46-7bff-fbd4-1409650da1cb
Article author
@dtivel
Metadata

ID: 4b107641-a326-0701-0406-c29627550fa5
Service: dotnet-fundamentals

Hi, @GaSkia.  Thanks for the feedback.
The default behavior of NuGet signed package verification (and the DOTNET_NUGET_SIGNATURE_VERIFICATION environment variable) on Linux only applies to NuGet restore operations.  .NET SDK install/update operations for dotnet workload and dotnet tool do not inherit NuGet's default restore behavior because they are not restore operations (e.g.: dotnet restore or dotnet build, which implicitly restores).  The dotnet workload and dotnet tool commands use NuGet packages, but the .NET SDK performs its own verification.  Perhaps documentation would be clearer if it said that dotnet workload and dotnet tool commands are not affected by NuGet's default restore behavior?
The .NET SDK team disabled automatic verification in their workload command via https://github.com/dotnet/sdk/pull/24590.  I believe they are planning on adding it back via https://github.com/dotnet/sdk/issues/37469.  However, this disablement/enablement is independent of NuGet restore operations.


Remove double backticks

Fixes #6213

I noticed a new warning on builds ...
Page is blank under monikers: aspnetcore-1.0, aspnetcore-1.1, aspnetcore-2.0, aspnetcore-2.1

Do I need to take action?

@guardrex No worries. I'm looking into that warning. It's related to a PR I was working on.


Adding Travis.ci

This repo needs signed up for travis, but the config is now here. Will also need to change the build badge URLs for redux-saga once redux-saga/redux-saga has been added to travis-ci
this addresses #1067

Thanks for the PR, great work :)


about pretrained mode to finetune

Hello,
I try to use the pretrained model that you provided to finetune, but when I use train script to set the ckpt, it tell me like this:
2021/07/08 20:26:17   File "train.py", line 377, in
Loading model from: /opt/conda/lib/python3.7/site-packages/lpips/weights/v0.1/vgg.pth
182021/07/08 20:26:17     G_A2B.load_state_dict(ckpt['G_A2B'])
192021/07/08 20:26:17 	Unexpected key(s) in state_dict: "encoder.stem.5.conv1.0.weight", "encoder.stem.5.conv1.1.bias", "encoder.stem.5.conv2.0.weight", "encoder.stem.5.conv2.1.bias", "encoder.stem.4.skip.0.kernel", "encoder.stem.4.skip.1.weight", "encoder.stem.4.conv2.2.bias", "encoder.stem.4.conv2.0.kernel", "encoder.stem.4.conv2.1.weight", "encoder.style.3.weight", "encoder.style.3.bias", "convs.6.conv.weight", "convs.6.conv.blur.kernel", "convs.6.conv.modulation.weight", "convs.6.conv.modulation.bias", "convs.6.activate.bias", "convs.7.conv.weight", "convs.7.conv.modulation.weight", "convs.7.conv.modulation.bias", "convs.7.activate.bias", "to_rgbs.3.bias", "to_rgbs.3.upsample.kernel", "to_rgbs.3.conv.weight", "to_rgbs.3.conv.modulation.weight", "to_rgbs.3.conv.modulation.bias".
202021/07/08 20:26:17 	size mismatch for convs.0.conv.weight: copying a param with shape torch.Size([1, 512, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([1, 256, 512, 3, 3]).
212021/07/08 20:26:17 RuntimeError: Error(s) in loading state_dict for Generator:
222021/07/08 20:26:17 	size mismatch for encoder.style.4.weight: copying a param with shape torch.Size([8, 512]) from checkpoint, the shape in current model is torch.Size([512, 8192]).
232021/07/08 20:26:17 	size mismatch for convs.0.activate.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256]).
242021/07/08 20:26:17 	size mismatch for convs.1.conv.modulation.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256]).
252021/07/08 20:26:17 	size mismatch for convs.2.conv.modulation.weight: copying a param with shape torch.Size([512, 512]) from checkpoint, the shape in current model is torch.Size([256, 512]).
262021/07/08 20:26:17 	size mismatch for convs.1.conv.modulation.weight: copying a param with shape torch.Size([512, 512]) from checkpoint, the shape in current model is torch.Size([256, 512]).
272021/07/08 20:26:17 	size mismatch for convs.2.conv.weight: copying a param with shape torch.Size([1, 256, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([1, 128, 256, 3, 3]).
282021/07/08 20:26:17 	size mismatch for convs.3.conv.weight: copying a param with shape torch.Size([1, 256, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([1, 128, 128, 3, 3]).
292021/07/08 20:26:17 	size mismatch for convs.2.activate.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128]).
302021/07/08 20:26:17 	size mismatch for convs.3.conv.modulation.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128]).
312021/07/08 20:26:17 	size mismatch for convs.4.conv.modulation.weight: copying a param with shape torch.Size([256, 512]) from checkpoint, the shape in current model is torch.Size([128, 512]).
322021/07/08 20:26:17 	size mismatch for convs.5.conv.weight: copying a param with shape torch.Size([1, 128, 128, 3, 3]) from checkpoint, the shape in current model is torch.Size([1, 64, 64, 3, 3]).
332021/07/08 20:26:17 	size mismatch for convs.5.activate.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64]).
342021/07/08 20:26:17 	size mismatch for convs.3.activate.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128]).
352021/07/08 20:26:17 	size mismatch for to_rgbs.0.conv.weight: copying a param with shape torch.Size([1, 3, 512, 1, 1]) from checkpoint, the shape in current model is torch.Size([1, 3, 256, 1, 1]).
362021/07/08 20:26:17 	size mismatch for convs.5.conv.modulation.weight: copying a param with shape torch.Size([128, 512]) from checkpoint, the shape in current model is torch.Size([64, 512]).
372021/07/08 20:26:17 	size mismatch for convs.4.conv.modulation.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128]).
382021/07/08 20:26:17 	size mismatch for to_rgbs.1.conv.modulation.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128]).
392021/07/08 20:26:17 	size mismatch for to_rgbs.0.conv.modulation.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256]).
402021/07/08 20:26:17 	size mismatch for to_rgbs.2.conv.modulation.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64]).
412021/07/08 20:26:17 	size mismatch for to_rgbs.1.conv.weight: copying a param with shape torch.Size([1, 3, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([1, 3, 128, 1, 1]).
422021/07/08 20:26:17 	size mismatch for to_rgbs.2.conv.weight: copying a param with shape torch.Size([1, 3, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([1, 3, 64, 1, 1]).
432021/07/08 20:26:17 	size mismatch for convs.2.conv.modulation.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256]).
442021/07/08 20:26:17   category=DeprecationWarning,
452021/07/08 20:26:17 	Missing key(s) in state_dict: "encoder.stem.4.conv2.0.weight", "encoder.stem.4.conv2.1.bias", "encoder.style.2.0.kernel", "encoder.style.2.1.weight", "encoder.style.2.2.bias", "encoder.style.5.weight", "encoder.style.5.bias".
462021/07/08 20:26:17 	size mismatch for convs.3.conv.modulation.weight: copying a param with shape torch.Size([256, 512]) from checkpoint, the shape in current model is torch.Size([128, 512]).
472021/07/08 20:26:17 	size mismatch for convs.4.conv.weight: copying a param with shape torch.Size([1, 128, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([1, 64, 128, 3, 3]).
482021/07/08 20:26:17 	size mismatch for convs.1.conv.weight: copying a param with shape torch.Size([1, 512, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([1, 256, 256, 3, 3]).
492021/07/08 20:26:17 	size mismatch for convs.1.activate.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256]).
502021/07/08 20:26:17 	size mismatch for to_rgbs.1.conv.modulation.weight: copying a param with shape torch.Size([256, 512]) from checkpoint, the shape in current model is torch.Size([128, 512]).
512021/07/08 20:26:17 	size mismatch for to_rgbs.2.conv.modulation.weight: copying a param with shape torch.Size([128, 512]) from checkpoint, the shape in current model is torch.Size([64, 512]).
522021/07/08 20:26:17   File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1045, in load_state_dict
532021/07/08 20:26:17 	size mismatch for to_rgbs.0.conv.modulation.weight: copying a param with shape torch.Size([512, 512]) from checkpoint, the shape in current model is torch.Size([256, 512]).
542021/07/08 20:26:17 Traceback (most recent call last):
552021/07/08 20:26:17 	size mismatch for encoder.style.4.bias: copying a param with shape torch.Size([8]) from checkpoint, the shape in current model is torch.Size([512]).
562021/07/08 20:26:17 	size mismatch for convs.4.activate.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64]).
572021/07/08 20:26:17 	size mismatch for convs.5.conv.modulation.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64]).
how can i fix this? maybe strict=False?

Try setting num_down=4


Features or requests for Alamofire 5?

Hi, I'm one of the developers of Alamofire. In addition to maintaining version 4, we're currently working on a major update, Alamofire 5. Given that Moya is one of the most popular libraries that consumes Alamofire directly, I thought I'd stop by and ask whether the users and library developers here had any feature requests or other changes they'd like to see in Alamofire.
Additionally, while we aren't too familiar with Moya, there are interesting features that may be suitable for integration into Alamofire itself. For example, Moya's more advanced request and response handling could be integrated in some way. So in addition to any features that may be needed, has anyone given any though about what features of Moya they might feel should be integrated directly into Alamofire?

Hey @jshier, thank you for taking the time to reach out!
Ironically, I was thinking last week that I need to get more familiar with the advanced capabilities of Alamofire and its internals to see if more enhancements can be made to Moya. I have no suggestions at the moment, but I'll tag @Moya/contributors to get some more ears on this topic 😃

Hi!
The first thing that popped to my mind was Codable.
I don't know if this already is on the roadmap for Alamofire 5.
I was thinking on an api that accepts a Codable object as parameter, so it will be easier to send json.
Something like this maybe?
struct Post: Codable { }

let newPost = Post()

Alamofire.request("https://httpbin.org/post", method: .post, parameters: newPost)

What do you think?? 😄

Back from when I was changing how Moya's response queue worked, I thought it would be awesome to be able to specify which queue to run an Alamofire request on; I would have loved to do something like have a serial queue for my network requests, managed by Rx.
I didn't look too far in to see if a client could specify which queue to have the work run on, if that isn't possible, I think having that control would be really cool 😄

I forgot to mention I have a GitHub (project board for Alamofire 5)[https://github.com/Alamofire/Alamofire/projects/1], so feel free to take a look and track progress as well as currently intended features.
@AndrewSB I'm currently prototyping a core rewrite of Alamofire that uses a single serial queue for all internal activities, including the URLSessionDelegate and *TaskDelegate. This has made threading much simpler and enabled some extra features (actual per-task errors for pinning failures), and should make Alamofire more scalable, since we wouldn't be creating so many queues all the time. I plan to make it a passable parameter. Once that's done, we'll do some profiling to see where we could benefit from separate queues (like request creation (now async) and response parsing) and go from there.
@kimdv That's something we'd like, but Encodable doesn't really map to request parameters very well. We're open to ideas though.

@jshier that sounds like a great potential perf win. Once you're done making the change, I'd love to see a method to pass in those queues, or variables that contain the queue names so clients (if they care about that level of granularity) can reach in and change or prioritize things

Alamofire 5 is now fully released! Sorry it took so long, but there were quite a few architectural questions to answers and APIs to plumb out. And, it turns out, writing documentation up to Mattt's standard is hard!
I still think it would be great to see more suggestions from Moya for new features, as well as any Moya features that might work better in Alamofire.
On a side note, I'll be starting a project at work that uses Moya, so I'll probably be around as we try to upgrade to Moya 14 / Alamofire 5.

Good stuff, Jon. Congrats on the release! 💪

hey @jshier, thank you for all the hard work on version 5! I think it goes without saying that when you rewrite the core of your project you need time to both QC and update the docs (especially given how big the Alamofire documentation is). Happy to see the final release!


how to configure print width to 120 characters? Thank You

Example:
Following are the results of the auto format
render json: {
               error: @user.errors.full_messages
             },
             status: :unprocessable_entity

I want the formatting results to work if the code we write exceeds 100 or 120 characters
like the following code:
render json: { error: @user.errors.full_messages  }, status: :unprocessable_entity

Are you trying to use Syntax Tree or the prettier print library?


[Android] Fix crash on file chooser

Fixed crash when no file is uploaded after the file chooser is launched.
Do not invoke RuntimeFileSelectHelper when the uploaded file is null.

BUG=XWALK-7278
BUG=XWALK-7271

@fujunwei @xzhan96 Please help to review

LGTM


No keybinds for luasnip choice selection

I have some snippets containing choice nodes and I've realized there is no  corresponding  key binds  specified for that action. I would like to know if we could add some key binds for that or at least mention in the config.

Oh, this is a really good point. :laughing: can't believe i forgot to add this

I actually tried to write something out for this, but think it's just easier to link and mention choice nodes in the doc. Added in 5ac4b58


It's ok to use Thrust in default cuda 8.0 runtime, but error occurs when use thrust in MFC or Win32 application.

When I program with visual studio 2013, it's ok to use Thrust in CUDA 8.0 runtime console applications, but problem would occur when I use thrust on MFC application.
Configuration of MFC/Win32 application:
link->input: cudart.lib
.cu file is set to compile with cuda/c++
error information:
Error	104	error C2027: use of undefined type 'thrust::detail::STATIC_ASSERTION_FAILURE'	C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\include\thrust\system\cuda\detail\for_each.inl	109	1
I really need your help for issue!
Or I have to use DLL complied with CUDA runtime.

do you observe similar behavior with 7.5? i'm not sure what is going on, but to help us to debug the issue please provide a self contained reproducer. thanks

I didn't try on 7.5, since I have a GTX 1080 GPU. I use the thrust sample code as follows, and add the code to a MFC/Win32 application, and set the right environment.
#include "cuda_runtime.h"
#include "device_launch_parameters.h"
#include <thrust/host_vector.h>
#include <thrust/device_vector.h>
#include <thrust/copy.h>
#include <thrust/fill.h>
#include <thrust/sequence.h>
#include
#include <stdio.h>
int thrustTest()
{
// initialize all ten integers of a device_vector to 1
thrust::device_vector  D(10, 1);
// set the first seven elements of a vector to 9
thrust::fill(D.begin(), D.begin() + 7, 9);
// initialize a host_vector with the first five elements of D
thrust::host_vector  H(D.begin(), D.begin() + 5);
// set the elements of H to 0, 1, 2, 3, ...
thrust::sequence(H.begin(), H.end());
// copy all of H back to the beginning of D
thrust::copy(H.begin(), H.end(), D.begin());
// print D
for (int i = 0; i < D.size(); i++)
std::cout << "D[" << i << "] = "<< std::endl;
return 0;
}
This program could run on 8.0 runtime, but for MFC/Win32, this would fail.
Thanks for your reply.

Is this still an issue for you?

Hi again. The CUDA backend for Thrust has been substantially rewritten in Thrust 1.9.0 (the version of Thrust in CUDA 9.0). The file you indicated you got an error in (thrust/system/cuda/detail/for_each.inl) is no longer there. Can you please try the CUDA 9.0 toolkit and see if you can reproduce your issue? Thanks!
When following up, please provide:

A minimal test case; e.g. a complete, standalone program that is as small as possible and has few/no dependencies on other software.
The entire sequence of commands/actions that reproduce the bug (using the minimal test case).
The unabbreviated outputs of those commands/actions including files, stdout/stderr and logs.
A description of the environment where the problem occurs (type and version of the hardware, OS, kernel, compiler and any other relevant software).

Closing. Please reopen with the requested info if this is still an issue.


docs(v2): showcase user QA-Board

Motivation
Hi,
We are working on a run-tracking framework for algo/perf engineering. We're using Docusaurus v2, and we're very happy about it. Thanks a lot for this project! I'm very impressed by how easy it is to get started, the number of features and how good it looks.
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
I ran the updated website and the showcase is updated with all links OK.
Related PRs
Not relevant.

@arthur-flam please sign the CLA. If you've done so, it's all good and we just need to wait for the bot to update this PR.

Nice to see a Samsung customer!

Very happy one 😄


Enable devmapper tests

Enables the devmapper tests back (after disabling them in https://github.com/containerd/containerd/pull/4719#issuecomment-730002886)
We have included a few patches recently that affected unit tests:

2b87d4554f08456b7b4476bf2d15a3dbd9083c26 adds a fix to pass tests after https://github.com/containerd/containerd/pull/4374
da686098668ec60eaca77dd82076044b5aad6698 partially reverts changes from https://github.com/containerd/containerd/pull/4235 (makes device removal deferred). @renzhengeek PTAL whether this still works for your case.

Signed-off-by: Maksym Pavlenko<EMAIL_ADDRESS>

LGTM


Add embed= support to API endpoints

Purpose
I am finishing up @samchrisinger's PR https://github.com/CenterForOpenScience/osf.io/pull/4260

JSONAPI defines a way to include related resources in a request (http://jsonapi.org/format/#fetching-includes). For us this means we could make a request like /api/v2/nodes/abcdef/?include=children to fetch a node and its children in a single requests.

Unfortunately the JSON-API spec has some drawbacks, so we're implementing a similar feature: embed. This directly embeds the view response for the related resource in the API response.

Changes

Implement JSONAPIBaseView to generically handle
injection of the request into the serializer context
injection of a dictionary of zero or more partial functions ( object -> dict) into the serializer context. These functions are called in the JSONAPISerializer's to_representation method when appropriate.
Add JSONAPIBaseView to existing view classes

Rewrite JSONAPIPagination (my changes)
By default, always embed node links and contributors

Make sure that embedded pagination links are for embedded resource, not parent resource:
url was http://localhost:8000/v2/users/?embed=nodes.  URL's link to user nodes list: Embeds parameter is not included for embedded resource.

When visiting second page of results, http://localhost:8000/v2/nodes/?embed=contributors&page=2, the page=2 filter does not extend to embedded resources:

When filtering, filtering not attempted on embedded resources.

Side effects

500 when embedding files from NodeProvidersList:

Minor issue: don't show the embed field in the representation of an embedded field.  It might give users the wrong idea.

Documentation:  there's an explanation of how embedding works in the Responses section of the Root docs.  Please move that up to the Requests section (perhaps after Pagination), and refer to it from the Response section.  Also, please document how to do multiple embeds.  I tried ?embed=children,comments and that didn't work, but ?embed=children&embed=comments did.  I don't think implementing the first version is necessary for this PR, we can do that down the road if folks want it.

I think the users relationship of NodeContributors is a candidate for always_embed.

?embed=nodes works on the UserList page, but not UserDetail:

@pattisdr this is looking really good!  Once you've fixed the issues above, please send it back to me in CR2, I'd like to take a look at the file endpoints.  Thanks for finishing this up and great work on getting the pagination links working!  I know Saman & Billy are really excited for this.

?embed=nodes works on the UserList page, but not UserDetail:

Good catch! I didn't check that endpoint. We were overriding get_serializer_context on UserDetail.

This is glorious. Onward, onward, to Ready-to-Merge!

:+1: Nicely done!

Failing tests are unrelated to this PR, and has been fixed on develop; no change needed.

Pass finished. Lookin close to merge!

Food for thought: since embeds are not part of the JSON API spec and are somewhat experimental, it might make sense to make this a proper extension.
The client would be required to send
ext=embed

in their Content-Type header.
This is something to consider for a public release but certainly not a blocker to merging this.

I've opened a new ticked for making this an extension: https://openscience.atlassian.net/browse/OSF-5245

I popped that into "winter sprint 1" @sloria

Looks good. I will make the last minor change to InvalidTargetError when I merge.

:tada:


[Bug]: Start up is stalling on "hash for docker build directory"

Is there an existing issue for the same bug?

[X] I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
[X] I have checked the existing issues.

Describe the bug
When I start up the most recent version of OpenDevin, it occasionally (but not always) gets stuck on the following steps and never goes to being usable:
===============================
16:22:48 - opendevin:INFO: runtime_build.py:125 - Input base image: nikolaik/python-nodejs:python3.11-nodejs22
Skip init: False
Extra deps: None
Hash for docker build directory [/var/folders/12/vk4h8csj4ts7rql173zqn0sm0000gn/T/tmp1be_5ull] (contents: ['Dockerfile', 'code']): 47664c37b4e2b65ff8034a05005d896b

Full log is here:
log.txt
Current OpenDevin version
commit bdf6df12c34bb944161c06f41eaef5864bf757d3

Installation and Configuration
make build; make run

Model and Agent
No response
Operating System
Mac OS Sonoma
Reproduction Steps
No response
Logs, Errors, Screenshots, and Additional Context
No response

I just started running test_runtime.py locally and run into similar (if not the same) issue.
In my case, the user creation inside the container errors because the user already exists:
22:30:01 - opendevin:INFO: runtime_build.py:272 - Image [ghcr.io/opendevin/od_runtime:47664c37b4e2b65ff8034a05005d896b] exists with matched hash for Docker build folder.
Will reuse it as is.
22:30:01 - opendevin:INFO: runtime.py:121 - Starting container with image: ghcr.io/opendevin/od_runtime:47664c37b4e2b65ff8034a05005d896b and name: opendevin-sandbox-test0aca0b87-3094-4bf1-8939-cb4ec22ba79b
22:30:01 - opendevin:WARNING: runtime.py:134 - Using host network mode. If you are using MacOS, please make sure you have the latest version of Docker Desktop and enabled host network feature: https://docs.docker.com/network/drivers/host/#docker-desktop
22:30:01 - opendevin:INFO: runtime.py:142 - Mount dir: /workspace
22:30:04 - opendevin:INFO: runtime.py:175 - Container started. Server url: http://localhost:52971
22:30:04 - opendevin:INFO: runtime.py:256 - Awaiting session
22:30:05 - opendevin:INFO: runtime.py:194 - Reconnecting session
22:30:05 - opendevin:INFO: runtime.py:200 -
------------------------------Container logs (last 10 lines):------------------------------
    |
------------------------------------------------------------------------------------------
22:30:09 - opendevin:INFO: runtime.py:194 - Reconnecting session
22:30:09 - opendevin:INFO: runtime.py:200 -
------------------------------Container logs (last 10 lines):------------------------------
    |             ^^^^^^^^^^^^^^
    |  File "/opendevin/code/opendevin/runtime/client/client.py", line 85, in __init__
    |    self._init_user(self.username, self.user_id)
    |  File "/opendevin/code/opendevin/runtime/client/client.py", line 148, in _init_user
    |    raise RuntimeError(
    |RuntimeError: Failed to create user opendevin: useradd: UID 1000 is not unique
    |
    |
    |ERROR:    Application startup failed. Exiting.
    |ERROR conda.cli.main_run:execute(125): `conda run PYTHONUNBUFFERED=1 poetry run python -u -m opendevin.runtime.client.client 52971 --working-dir /workspace --plugins agent_skills jupyter --username opendevin --user-id 1000` failed. (See above for error)
    |```

Hmm, maybe the previous version just had stalled as a zombie process. I’ll close this.

@tobitege I think your issue is different, maybe see here:
https://docs.all-hands.dev/modules/usage/custom_sandbox_guide#error-useradd-uid-1000-is-not-unique

Actually, nevermind, it is indeed a problem still (even after killing the zombie process, deleting the docker containers, and installing from scratch).

@tobitege I think your issue is different, maybe see here: https://docs.all-hands.dev/modules/usage/custom_sandbox_guide#error-useradd-uid-1000-is-not-unique

Thanks, but that sandbox_user_id="1001" in the toml isn't picked up, because the error in my above post is happening inside the container

OK, maybe we can open a separate issue for that one.


[Suggestion] Heat from gas and liquid fuels

Generating steam from charcoal is nice and all but I would like to be able to use ethylene and/or lava to produce steam as a mid-game power source. It would be nice if it could also use gas and liquid fuels from other mods.

This is planned, actually :)

Yey! :D


compilation error:extern_lib.h:64:14: error: no template named 'map' in namespace 'std'; did you mean 'max'?

My configuration: ubuntu 22.04 python 3.10 gcc  11.2.0
When running make,a lot of erors got pump out:
extern_lib.h:64:14: error: no template named 'map' in namespace 'std'; did you mean 'max'?
pass.cc:55:12: error: cannot initialize a parameter of type 'llvm::Pass *' with an rvalue of type 'llvm::ModulePass *'
llvm.cc:43:10: fatal error: 'llvm/Support/TargetRegistry.h' file not found
selection/generator.cc:334:69: error: no matching member function for call to 'CreateGEP'
generator.cc:3328:14: error: no member named 'addAttribute' in 'llvm::Function'

Looks odd. Probably because you were not using the LLVM version that triton supports?

I think this should be fixed now that we package LLVM in. Let us know if the issue hasn't disappeared :)


B14.0 software release

💡 Description

Release stable version for PDS EN tools
roundup releases (github, pipy, maven)
(first checkbox = released, second checkbox = in the software summary)

[x]  [ ] pds4-information-model 14.3.0
[x] [ ] pds4-jparser 2.6.0
[ ]  [ ] validate, 3.3.0
[ ]  [ ] registry-common
[ ]  [ ] harvest (update registry-common dependency)
[ ]  [ ] registry-mgr (update registry-common dependency)
[ ]  [ ] registry-loader (update harvest & registry-mgr dependency)
[ ]  [ ] registry-harvest-service
[ ]  [ ] registry-crawler-service
[ ]  [ ] registry-harvest-cli
[ ]  [ ] registry-sweeper
[ ]  [ ] registry-api
[ ]  [ ] registry,
[ ]  [ ]  pds-api,
[ ]  [ ] pds-api-client
[ ]  [ ] doi-service
[ ]  [ ] deep-archive
[ ]  [ ] MILabel
[ ]  [ ] PLAID
[ ]  [ ] Transform
[ ]  [ ] Feedback widget
[ ]  [ ] upload-data-manager
[ ]  [ ] nucleus

Javascript/web releases:

[ ]  [ ] doi-ui
[ ]  [ ] wds-react

Docker hub releases

[ ]  registry-loader

[ ]  registry-harvest-service

[ ]  registry-crawler-service

[ ]  registry-harvest-cli

[ ]  registry-api

[ ]  Software release summary

@tloubrieu-jpl not sure if you tried to tag validate? But looks it did not work. I was seeing lots of weird issues when trying to tag it.
Also looks like you had lots of stale SNAPSHOT tags locally that got pushed.

Software summary is published on https://nasa-pds.github.io/releases/14.0/index.html


Opening zip-file launches the Zoom-App

Trying to open an attached zip-file launches the Zoom-App. Presumably some other app if Zoom is not installed.
To Reproduce:

Send yourself a zip file
Open the email in K9 (Android version)
Tap on the "Open" button for the attachment.
-> K9 launches Zoom
Have a heart attack as you think K9 has executed the contents of a zip-file rather than letting you see the contents.

Expected behavior:

Either: Let me choose the app to use to open the zip-file: I have no app for zip-files, but several generic viewers that show up whenever unknown extensions are found
Or: list the contents of the zip-file in K9 itself and offer to open or download those files

Environment:

K-9 Mail version: 5.600
Android version: 9
Device: BlackView 9900
Account type: IMAP

Additional context:
Zoom is installed, of course.
Log file attached.
k9-log.txt

When clicking an attachment K-9 Mail asks the system to open the file for viewing. It's up to the system to find a matching app.
The behavior you're seeing is most likely caused by the Zoom app doing things it shouldn't do.


Improvement suggestions for poisons

(I got those in a DM)

The handling of poisons with Onset times (most Ingested poisons) leaves a bit to be desired.

Consider Belladonna as an example.

When you first ingest Belladonna, you roll a Fortitude save. Assume you failed, and are subject to the Belladonna affliction. So you use Extempore Effects to apply it as an effect on the character.

Extempore Effects applies an effect that shows you at Stage 1, with an expiration of 10 minutes
This is wrong - you don't start in Stage 1, because there's an Onset period before anything takes effect
Instead, there should be 4 phases (Onset, Stage 1, Stage 2, and Stage 3), and you should start at the Onset stage

When you advance the world clock by 10 minutes, the effect gets marked as Expired (and we'll assume that the system automation isn't set to auto-remove it). To continue tracking, you need to open up the sheet, and extend the duration by another 10 minutes (increasing it from 10 minutes to 20 minutes) to un-expire it. This is a bit unintuitive if you aren't used to re-activating Expired effects, which I was not. It's easy enough to roll saves and apply stage-based effects here, because they're right there in the effect sheet and easily clickable, but adjusting the duration and stages each time is slow.

I don't know if this is even possible, but generating the effect description with embedded macros that could be clicked to advance or roll back the stage would be helpful.
The macro to advance the stage would ideally increment the stage counter, and adjust the duration based on the new stage
The macro to roll back the stage would decrement the stage counter, and also adjust the duration
Both macros would ideally check if the maximum duration has been reached (respecting that time set in the Onset phase does not count towards the maximum), and refuse to extend the duration further if so (and notify of this)

In a test with Giant Centipede Venom, the initial duration set on the effect doesn't seem to match what's described in the Readme, and what I saw with Belladonna. As described, the initial duration is said to be set based on the duration of Stage 1 - but in this case, it applied a duration of 6 rounds. This is the maximum duration of Giant Centipede Venom, not the Stage 1 duration. Now, the max duration is MUCH more useful for injury poisons with single round stages, so I do prefer this - but it's a mismatch between description and reality.

Overall, the injury poisons in combat feel smoother than the longer term poisons.

I think what I could do is:

when there's an onset stage, add "onset" as a stage (so it's:  Onset, Stage 1, Stage 2..., and starts at Onset)
for long-term afflictions:  add an extra "Advance stage forward" button to the description, which moves to the next stage and sets the duration to be that stage's duration.  then another button that does the same but backwards (one step down).  The effect should un-expire when this happens -- which might require me to change its start time in the data.

if not buttons, macros can work too

+1 to this idea, might look into a possible PR or something to help as it's been something I've been toying around with on my own as well


bug: #511 (3971ec5) makes 'ctrl-d' key (delete buffer) not working in Windows

https://github.com/linrongbin16/fzfx.nvim/commit/3971ec58038fa8a1fbddc7e440c805ecc61538ed


Fatal error: Allowed memory size of ... / change memory limit in binary

Hello,
I am using Castor 1.15.0 to make a small test program that imports 2 functions but I have a Memory Limit error on the binary version
vendor/bin/castor repack --app-name=wixi

vendor/bin/castor compile wixi.linux.phar

How can I modify the embedded PHP configuration to lift this limit?

For the moment the only way i see this is having something like this
ini_set('memory_limit', '256M');

At the start of your castor.php file

I don't know why but I don't have the problem anymore.
I didn't change the value with "ini_set" and the binary version no longer has a "memory_limit" since my build this morning. 🤷


404 Error

I have tried to create a local tunnel link but the link always gives me the 404 error.

Me too. Did you resolve this?

Me too. Did anyone resolved?

It worked for me when I was outside of my company's network. So I assumed it was firewall restrictions causing it to not work.

Thanks a lot @jstrugnell 😄
You saved my time ❤

Glad I could help :smiley: :thumbsup:

I'm super confused. I'm trying to use LocalTunnel as part of BrowserSync within Laravel: https://laravel.com/docs/5.6/mix#browsersync-reloading
I see on the LocalTunnel homepage "Localtunnel allows you to easily share a web service on your local development machine without messing with DNS and firewall settings."
So why would I ever get this error message that tells me to check my firewall?
      Tunnel: https://kvbss.localtunnel.me
 -----------------------------------------
          UI: http://localhost:3001
 UI External: http://<IP_ADDRESS>:3001
 -----------------------------------------
[BS] Watching files...
[BS] Localtunnel issue: connection refused: localtunnel.me:36626 (check your firewall settings)
[BS] Oops! The localtunnel appears to have disconnected. Reconnecting...

(And why are the comments above in this ticket about firewalls if the point of LocalTunnel is to not need to worry about what the firewall settings are?)
Like the original poster of this ticket, all I see at my site (https://kvbss.localtunnel.me) is "404".
Please forgive my naivete; I'm completely new to LocalTunnel and just have never gotten it to work yet.
Thanks in advance for any suggestions.

That error message can sometimes be misleading. It basically means the localtunnel client was not able to connect to the localtunnel server. Because of the way localtunnel client and server interact (high numbered tcp ports) these are sometimes blocked by firewalls.
The 404 when visiting the site means that localtunnel does not know about your tunnel connection (and thus there is no page for it to display).

Why not to ReOpen this issue since it happens almost always?
Error: connection refused: localtunnel.me:40049 (check your firewall settings)
    at Socket.<anonymous> (C:\Users\Me\AppData\Roaming\npm\node_modules\localtunnel\lib\TunnelCluster.js:47:32)
    at Socket.emit (events.js:182:13)
    at emitErrorNT (internal/streams/destroy.js:82:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)
localtunnel server crashed
your url is: https://me.localtunnel.me
C:\Users\Me\AppData\Roaming\npm\node_modules\localtunnel\bin\client:65
        throw err;
        ^

Error: connection refused: localtunnel.me:42372 (check your firewall settings)
    at Socket.<anonymous> (C:\Users\Me\AppData\Roaming\npm\node_modules\localtunnel\lib\TunnelCluster.js:47:32)
    at Socket.emit (events.js:182:13)
    at emitErrorNT (internal/streams/destroy.js:82:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)
localtunnel server crashed

@laukstein Did you find a way to fix this problem? Thank you.

@holdenla, no, I stopped using localtunnel. Must localtunnel issues are caused by localtunnel server downtime, can't relay on it.

@laukstein In meantime I tried using ngork and I get the same error . Not sure if it is because of the firewall, but I need to find a way to test my project from localhost on web.

It worked for me when I was outside of my company's network. So I assumed it was firewall restrictions causing it to not work.

Me too. Then I really don't know the propose of localtunnel if it doesn't work in these scenarios...

me too. Do anyone knows an working alternative to localtunnel.

I'm also receiving the 404 error so I started using Ngrok.

Hey all, if you get a plain 404 page, it usually means the tunnel dropped (so eithe rthe server or client disconnected or ISP issues).

Is there any way to resolve this issue?

Usually restarting the client on your side should fix it

i did
sudo apt install npm
and then
sudo npm install -g localtunnel

i did
sudo apt install npm
and then
sudo npm install -g localtunnel
then
lt --port <_whatever port_> -s <_whatever subdomain_>
or if youre using a tool just use localtunnel

i did
sudo apt install npm
and then
sudo npm install -g localtunnel
then
lt --port <_whatever port_> -s <_whatever subdomain_>
or if youre using a tool just use localtunnel

i did
sudo apt install npm
and then
sudo npm install -g localtunnel
then
lt --port <_whatever port_> -s <_whatever subdomain_>
or if youre using a tool just use localtunnel

i did
sudo apt install npm
and then
sudo npm install -g localtunnel
then
lt --port <_whatever port_> -s <_whatever subdomain_>
or if youre using a tool just use localtunnel


pull request for e2e test at 11/10/2016 20:15:09

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

optest/docs/updated_at.md
:white_check_mark:Succeeded

Details

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


get ride of all OS non agnostic code - use Mage instead

saw the ps1 fiel for the client.
You can use Mage to get ride of all this !
https://github.com/magefile/mage
Mage is sort of like Makefiles but for golang.
You can also use them non compiled, but then compiel the mage file to have it non interpreted (not the right term i know, but you get the idea ) and not need golang installed.
Lost of golang programs using it.
Some great examples here:
https://github.com/mcandre/mage-extras
There are many other examples around.
Will be pretty quick to convert the stuff too.

Yep, I'm feeling good about mage.
On Fri, 7 Dec 2018 at 22:49, gedw99<EMAIL_ADDRESS>wrote:

saw the ps1 fiel for the client.
You can use Mage to get ride of all this !
https://github.com/magefile/mage
Lost of golang programs using it.
Some great examples here:
https://github.com/mcandre/mage-extras
There are many other examples around.
Will be pretty quick to convert the stuff too.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/shenzhen-go/issues/36, or mute the thread
https://github.com/notifications/unsubscribe-auth/ACSXrOvOjPiw01R2UdrNTo2U5wDf0H7nks5u2lW1gaJpZM4ZITOF
.


Issue 41749: Importing wikis via MemoryVirtualFile now fails

Rationale
Issue 41749: Importing wikis via MemoryVirtualFile now fails

Verifying fix: https://teamcity.labkey.org/buildConfiguration/LabKey_2011Release_Community_BvtAPostgres/1163258

@labkey-tchad That suite is all green

My DailyB suite run also passed


Add storage helper and migrate config entries

Description:
This adds a storage helper. This will make it easier for components and core pieces to store and load data.

Stored data needs to be JSON serializable. However when and where the config is stored is opaque to the developer. This leaves options open for the future 👍
Optionally add a write delay in case you expect a lot of writes
If delayed write is scheduled, load will return correct config
If delayed write, shutting down Home Assistant will trigger the write
Add a migrator from old formats so that it helps existing code migrate to the storage helper.
Track versioned data and migrate if necessary

Since I did not want to add logic that is unused, I've migrated the config entries logic to use the new storage helper.
Bonus:

Fixes https://github.com/home-assistant/architecture/issues/37#issuecomment-397961432. Adds a new hass.async_add_executor_job if we know it's going to be a sync method. This will be the new preferred way for sync methods. Will help step towards https://github.com/home-assistant/home-assistant/pull/14429#discussion_r188084991

Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass

If the code does not interact with devices:

[x] Tests have been added to verify that the new code works.

Future: It would be nice to add a voluptuous schema for data partition. That make it easy to add defaults or changes over the schema.

I don't want to include voluptuous, the idea is that this is written/read by code, not humans. If defaults change, they can do a one time migration to create those. It should not run on every run.

Yeah read/write by code. That mean you need inheret this class and write migration function and need holt this until you are sure that no user use the old version.
I use voluptuous on supervisor and this allow to downgrade from version 100 to 5 back to 15 and agen to 100 without a break of functionality. That is why I love voluptuous. And on website of voluptuous stay: Voluptuous, despite the name, is a Python data validation library. It is primarily intended for validating data coming into Python as JSON, YAML, etc.
Anyway, I had think that we use allready this magic tool and have good expirense, so we can use this on begin for new stuff instead to make a big migration like service stuff.

I don't want to register the STOP event during initialization because it means that startup of Home Assistant is slower. With the current logic, although it looks a bit complicated, we make sure that we only register handlers when we need it and and as little as possible.


Better default lane routing

Copied from original title:

AI should automatically pick the furthest outside lane when turning right from right lane, or furthest inner lane when turning left from furthest left lane (US, Right hand Roads)

Describe your idea
What is set up in the road should be automatically set up, but invisible to the lane arrows menu. it should be at an AI level so that the cars TRY to do that, but if they fail to, then they revert back to the default AI for that intersection.
it would also be nice if cars automatically merged into the furthest lane on the side that the onramp is on.
it would also be cool to have automatic intersection laws for certain combinations of roads. like if a 2 lane crosses a dirt road, you set it to automatically place stop signs on the dirt road, and automatically make the 2 lane big road have the right of way.
like, what's in the screenshot should automatically be set up, or it should be the default thing the cars try to do
Screenshots?

Notes or questions?

Tagging #706

I totally agree.
So like highway merging rules cars should PREFER merging with outer lanes. the problem with highway rules is that its too aggressive and too restrictive.

it would also be cool to have automatic intersection laws for certain combinations of roads. like if a 2 lane crosses a dirt road, you set it to automatically place stop signs on the dirt road, and automatically make the 2 lane big road have the right of way.

In sim city if a small road joins a main road, the small road gets an stop sign and only takes short turn. we should do the same in CS.

we need to give user the option to setup intent. default traffic rules would match user intent but can be overridden
.- for simple cases (not too aggressive) we should auto-detect intent like in highway merging rules (only when lane arithmetic works), small roads merging with main road, simple straight roundabout, ...

Just my ideas.

Would be good if we could test it out in something like #706 and maybe #631, so that users are manually applying it to start with - that way we will quickly learn of any bugs and quirks. Once proven reliable, we can then add some mod options that allow it to be automatically applied.

@aubergine10 My thoughts too. little by little. Road selection panel is a good way to express intent in future . although we need to express intent on junctions too.
My secret plan all along has been (not so secret any more!):

implement quick setup
let users to express intent with the possibility to override them.
automatic detection of simple intents with the possibility to override them.

This was my secret plan too! Although I had an additional step:

Add a "Biffa my city!" button that automatically configures all roads in the city. 🤣

@kianzarrin Is this issue complete now that #706 and #631 are merged, or is there still extra stuff to do?


ASP.Net Core (preview) test task failing to run the found tests

Hi again,
Now that my build is working I am investigating the reason why my tests are not executed.
logs_109.zip
As can be seen on line 3185 in the test log, it found 2 test assemblies, and 1 testadapter (line 6219).
However, it does not execute any of the tests.  I have tried both debug and release builds with no difference.
I notice that the path in the log specifies d:\a\3\s\Dev\Jbssa.MasterData\tests\Jbssa.MasterData.UnitTests\bin\Release\netcoreapp1.1\Jbssa.MasterData.UnitTests.dll
where the physical path is actually:
d:\agent\_work\3\s\Dev\Jbssa.MasterData\tests\Jbssa.MasterData.UnitTests\bin\Release\netcoreapp1.1
but I'm assuming that it is using a path alias??? or something, since the build set uses a similar path ProjectFile: 'd:\a\3\s\Dev\Jbssa.MasterData\Jbssa.MasterData.sln'
Can anyone shed some light on why my tests are not being executed?

@Allann Two things:

searchFolder=d:\a\3\s
Have you set this search folder in task definition. Check if you have used some variable which is having incorrect value? Aslo have you added any smbolic link,  as I can see "findOptions.followSymbolicLinks: 'true'"?

How are you passing the test adapter path?  And what happens if you run the command printed in logs manually via cmd?
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" d:\a\3\s\Dev\Jbssa.MasterData\tests\Jbssa.MasterData.UnitTests\bin\Release\netcoreapp1.1\Jbssa.MasterData.UnitTests.dll d:\a\3\s\Dev\Jbssa.MasterData\tests\Jbssa.MasterData.UnitTests\bin\Release\netcoreapp1.1\xunit.runner.visualstudio.dotnetcore.testadapter.dll /EnableCodeCoverage /logger:trx "/TestAdapterPath:"d:\a\3\s""

@Alann Did you get the chance to try to check points mentioned in previous comments. Thanks.

@vimegh - sry i missed the response above, will check my spam filters.

I can't use that search folder, if i do it won't run on the different agents we use (local and hosted). Currently we use ($(System.DefaultWorkingDirectory)
I'm not.  There is a field called Path to custom test adapters. But the info button says Nuget restored packages are automatically searched for.  Since xUnit is installed via NuGet I didn't think I needed to add anything here.
Further I don't have permission to log into the local build server to run the commands locally. However, running them via a similar command line in my own environment works fine. Obviously the paths are different in this instance.

As additional information, this is the default test task added by the ASP.NET (PREVIEW) template.  It is easy to reproduce the above issue by creating a new ASP.Net Core web application and then add a xunit test project.
I can't use the ASP.NET Core build template because I can't set the credentials for external collections (i.e. telerik.com).  If this is not the case and this can be added to that template, can you please give instructions as to how this is achieved.
The steps in the current template:
NuGet restore:

NuGet 2.*
Command: restore
Path to solution: **/*.sln
Feeds: Feeds in my nuget.config
Path to Nuget.config: $Master Data/Dev/Jbssa.MasterData/nuget.config
Credentials for feeds: telerik.com
Build solution:
Version 1.*
Solution: ***.sln
Visual Studio Version: latest
MSBuild arguments: /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactstagingdirectory)\WebApp.zip" /p:DeployIisAppPath="Default Web Site"
Platform: $(BuildPlatform)
Configuration: $(BuildConfiguration)
Test Assemblies:
Version 2.*
Select tests using: Test assemblies
Test assemblies: $(BuildConfiguration)\netcoreapp1.1*test.dll !*\obj**
Search folder: $(System.DefaultWorkingDirectory)
Test filter criteria: blank
Run only impacted tests unchecked
Run Test mix contains UI tests unchecked
Select test platform using: Version
Test platform version: Latest
Settings file: blank
Code coverage checked
Build platform: $(BuildPlatform)
Build configuration: $(BuildConfiguration)

Hope this helps clarify a little

Thanks @Allann for sharing the details. Assigning to Tanvi to check it out.

@Allann The VSTest task currently doesn't support dot net core tests. You will need to use the dotnet core task with the command as test . You can add a publish test results task to publish the results back to the service. There is an improvements planned here with respect to results publish. The dotnet core task will do an auto publish of the results with the command as test.

@cltshivash thanks for getting back to me on this, i'll try to add that tomorrow when i'm back at work.
Just as an aside, I find it strange that a ASP.Net Core solution template which includes tests doesn't actually support the tests project. Maybe that is just me?  Is there an expected delivery on this?
To prevent others from having the same issue, shouldn't the ASP.Net Core (preview) template have the test task removed since it doesn't support tests?


Senior Childhood Photos & Graduation Video

MCA community-wide email launched today. Currently we have 48 seniors:
Dear Senior Families,
It is that time of year when we reach out to senior parents to collect childhood photos of our current seniors.  These childhood photos will be used for a slideshow presentation during the MCA Class of 2024 Graduation Ceremony.  We would like to have 15-20 photos of your child to choose from, from infant through present. Thank you so much for your help in collecting these!  It will help us make a very special presentation for the graduation ceremony!
To submit, please scan photos individually. The photos must be in .jpg or .png format and ATTACHED to the email, not inserted or embedded in the email.  Please email all photos to<EMAIL_ADDRESS>The deadline for these is Monday, January 8th. Thanksgiving Break is a great time to look through old photos to make selections!  Thank you for your assistance, and we can't wait to see these photos!
Sincerely,
MCA Admin

Please add information on what we do with the photos (store them on Google Shared drives, etc), any important dates to keep an eye on, and what are the main items we're responsible for in this and where they're located or going to be located.

I collect, rename and organize these pictures in my google drive as they flow in to me from parents. I begin producing the video in the spring after PM2. Graduation is on May 24th. I anticipate having the sample available to Mr. Hull by May 10th.

this is a really cool idea. Let me know if you want any help with this project. I know everyone will be excited to see it.

Pictures coming in slowly. I need to send a 'deadline' email out I suppose. So far, only three of forty-eight parents have submitted.

Told Gena I will make personal calls to parents who've not submitted photos on our return from break. I'll email them personally too.


🌱 Bump CAPI to 1.6.6

What this PR does / why we need it:
Bump CAPI to 1.6.6

/test metal3-ubuntu-e2e-integration-test-release-1-6

/lgtm


Add support for loading data from pandas dataframe

Right now, we only support loading data from numpy arrays. It would be nice if there was a helper function to convert a dataframe of scores to numpy arrays. Some initial code to help what this might look like:

def get_all_return_values(df):
  games = list(df['game'].unique())
  return_vals = {}
  for game in games:
    game_df = df[df['game'] == game]
    arr = game_df.groupby('wid')['normalized_score'].apply(list).values
    return_vals[game] = np.stack(arr, axis=0)
  return return_vals

def convert_to_matrix(x):
  return np.stack([x[k] for k in sorted(x.keys())], axis=1)

## Usage
# Array of shape (num_runs, num_games, num_steps)`
all_normalized_scores = convert_to_matrix(get_all_return_values(score_df))

The above code assumes we have a pandas Dataframe with keys run_number, 'gameandnormalized_score` containing scores for all steps (in a ordered manner).

Hi, just to better understand the assumed structure of the DataFrame: We have one row, for each step?
Are these all the steps during evaluation (not training) on all the tasks?
And we'd assume separate DataFrames for each approach, which are each read separately by get_all_return_values()?
Eg, to construct the required dict for computing performance profiles.

Yeah, for performance profiles, the data frames contain per-step results from evaluation (obtained during the course of training).
For aggregate metrics, we use the final performance, so that corresponds to evaluation results at the final step or a pre-specified step.


Subtitle / CC can't be turned on

As per subject, the subtitle / CC button can't be turned on. It just loads and loads and loads but nothing happened. I'm in FF 35, GM 2.3, YTC script Dev Build 4.81. I want to roll back to previous version but don't know how T__T Pls anyone help me...

You can find older versions here.

The subtitle button finally can be turned ON. I don't know whether other people also experienced this. If nobody does, pls close this issue.

I'm not sure where the issue could have come from, but if it's not happening then I will close this issue. Also if your issue does appear again then create a new issue with a reference to this issue.


How dose triton support  control divergency on compute？

Triton only support masks on some memory-related operations, such as load/store.  In some scenarios, we needs control divergence on compute operations, for example: hash map algorithm, which needs masks when compute. So how does triton support control divergence on compute? Is there a roadmap?

if/else clauses are supported in triton. You can also use tl.where to set elements based on some condition.

there is no warp divergence in Triton because if/else take in scalar values that are shared by all threads in the block. As mentioned above, you can use tl.where to mask elements of a tensor based on a condition, which is also safe because it doesn't use any branching under the hood

Thanks for relay! @ptillet @CHDev93


Speckled Documents Create Psychological Case for Tesseract

Tesseract just spent seven hours trying to do OCR on the attached document. It's five pages long.
I'm fairly certain that the reason this takes so long is because of the speckling in the document. Other times when I've seen this kind of performance, it's been for similarly speckled documents.
Not sure what you can or should do about it, but since it seems to be a worst case scenario for Tesseract, I thought I'd report it.
This is on the latest version of Tesseract.
gov.uscourts.ctd.18812.88.0.pdf

Tesseract just spent seven hours trying to do OCR on the attached document. It's five pages long.

Wow, This is really extreme!

Well, I tested it and it takes less then 4 minutes...
Tesseract (the official command line tool) does not excepts pdf as input, so how did you converted the pdf to a format that Tesseract excepts?
Here is what I did:
convert gov.uscourts.ctd.18812.88.0.pdf gov.png'

This command will create 5 'gov-n.png' images.
First page:
tesseract gov-0.png gov-0

time: 1 minute and 5 seconds

One minute per page is not extraordinary much (although improvements which make it faster are of course welcome). My worst cases are currently double pages from a historic newspaper which take around ten minutes.

Thanks for looking at this! We converted using ghostscript to multi-page tiff:
gs -dQUIET -dSAFER -dBATCH -dNOPAUSE-sDEVICE=tiffgray -r300x300-o destination path

You could use gs to split the pdf into images and then ocr each separately
and concatenate the result.
On 23 Sep 2016 10:58 p.m., "Mike Lissner"<EMAIL_ADDRESS>wrote:

Thanks for looking at this! We converted using ghostscript to multi-page
tiff:
gs -dQUIET -dSAFER -dBATCH -dNOPAUSE-sDEVICE=tiffgray -r300x300-o
destination path
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/tesseract/issues/431#issuecomment-249253159,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o7sVanef-bvL1nJdyJAdBJ0L3-2jks5qtAwkgaJpZM4KEcsN
.

Sure, but that's not the point...and anyway, it's not at all clear that the slowness is because it's a multipage tiff. I suspect if you ran this on each individual page of the tiff you'd have the same slowness.

To get accurate results, you will need to preprocess the images too to get
rid of the background speckles.
You could try scantailor or imagemagick.
As a test, you can also try Vietocr GUI, and compare results with the
command line output.
On 23 Sep 2016 11:35 p.m., "ShreeDevi Kumar"<EMAIL_ADDRESS>wrote:
You could use gs to split the pdf into images and then ocr each separately
and concatenate the result.
On 23 Sep 2016 10:58 p.m., "Mike Lissner"<EMAIL_ADDRESS>wrote:

Thanks for looking at this! We converted using ghostscript to multi-page
tiff:
gs -dQUIET -dSAFER -dBATCH -dNOPAUSE-sDEVICE=tiffgray -r300x300-o
destination path
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/tesseract/issues/431#issuecomment-249253159,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o7sVanef-bvL1nJdyJAdBJ0L3-2jks5qtAwkgaJpZM4KEcsN
.

Yeah, we saw this in testing, but went with TIFFs because they support multi-page images, which makes our OCR pipeline easier. In testing, we saw that the OCR for PDFs was no slower using large TIFFs than it was using PNGs because the process seems to be CPU bound no matter what.
If you use 300dpi PNGs do you get the slow performance I experienced with the 300dpi TIFFs? That's probably a better test, right?

Image properties:
Width: 35.417 Height: 45.834
DPI: 72 X 72
This is equivalent to:
Width: 8.5 Height: 11
DPI: 300 X 300

gs -dQUIET -dSAFER -dBATCH -dNOPAUSE -sDEVICE=tiffgray -r72 -o gov.tiff gov.pdf
OR
gs -dQUIET -dSAFER -dBATCH -dNOPAUSE -sDEVICE=tiffgray -o gov.tiff gov.pdf
This command creates a 42 MB tiff file. The size in pixels of each page is the same as with my PNGs.
It takes 4 minutes and 29 seconds to Tesseract to read this tiff.

@Shreeshrii commented,

To get accurate results, you will need to preprocess the images too to get
rid of the background speckles.

I'm guessing that it will run faster too.
BTW, Here is what Tesseract outputs in the console:
time tesseract gov2.tiff govtiff
Tesseract Open Source OCR Engine v3.05.00dev with Leptonica
Page 1
Detected 1875 diacritics
Page 2
Detected 1338 diacritics
Page 3
Detected 1885 diacritics
Page 4
Detected 658 diacritics
Page 5
Detected 213 diacritics

real	4m29.118s
user	4m28.972s
sys	0m0.152s

It 'thinks' the speckles are diacritics...

Thanks for looking at this @amitdo.

This command creates a 42 MB tiff file. The size in pixels of each page is the same as with my PNGs.

But these aren't 300x300, which is apparently what provides the best OCR quality.[1] The point of this issue is that at 300x300, this takes seven hours to do five pages.

It 'thinks' the speckles are diacritics...

Yeah...that's an issue too. Running a despeckling filter first would help in this case, but we do OCR on millions of PDFs and we only need to despeckle the worst of them. For the rest, I imagine it would reduce quality (not to mention slow down the pipeline).
The point here is that Tesseract takes seven hours for a speckled document at the recommended DPI.
[1]: Some references:

From the FAQ: https://github.com/tesseract-ocr/tesseract/wiki/FAQ#is-there-a-minimum-text-size-it-wont-read-screen-text
"Optimal Image Conversion Settings for Tesseract" : https://mazira.com/blog/optimal-image-conversion-settings-tesseract-ocr
"Using Tesseract with PDF scans": http://kiirani.com/2013/03/22/tesseract-pdf.html

Since the document is in English, another test can be to ocr it with
tesseract 3.02 and compare performance to see  that there is no regression.
On 24 Sep 2016 3:18 a.m., "Mike Lissner"<EMAIL_ADDRESS>wrote:

Thanks for looking at this @amitdo https://github.com/amitdo.
This command creates a 42 MB tiff file. The size in pixels of each page is
the same as with my PNGs.
But these aren't 300x300, which is apparently what provides the best OCR
quality.[1] The point of this issue is that at 300x300, this takes seven
hours to do five pages.
It 'thinks' the speckles are diacritics...
Yeah...that's an issue too. Running a despeckling filter first would help
in this case, but we do OCR on millions of PDFs and we only need to
despeckle the worst of them. For the rest, I imagine it would reduce
quality (not to mention slow down the pipeline).
The point here is that Tesseract takes seven hours for a speckled document
at the recommended DPI.
[1]: Some references:

From the FAQ: https://github.com/tesseract-ocr/tesseract/wiki/FAQ#is-
there-a-minimum-text-size-it-wont-read-screen-text
https://github.com/tesseract-ocr/tesseract/wiki/FAQ#is-there-a-minimum-text-size-it-wont-read-screen-text
"Optimal Image Conversion Settings for Tesseract" :
https://mazira.com/blog/optimal-image-conversion-settings-tesseract-ocr
https://mazira.com/blog/optimal-image-conversion-settings-tesseract-ocr

"Using Tesseract with PDF scans": http://kiirani.com/2013/03/22/
tesseract-pdf.html

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/tesseract/issues/431#issuecomment-249310613,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o8Nb1itUTsqp1oq0j2O5w7qLdc0Yks5qtEkwgaJpZM4KEcsN
.

This PDF file is just a bag of images. This is very common and was probably produced by
a photocopier or sheetfed scanner. Some fax machines make these too. It is entirely
black and white.  This command renders 100%
equivalent images for 2.3MB.
gs -dQUIET -dSAFER -dBATCH -dNOPAUSE -sDEVICE=tiffg4 -r300x300 -o gov2.tiff gov.pdf
That said, best practice for known 'bag of images' PDF is not to render anything. It is to
extract the images, undisturbed. If necessary, adjust their header so that their resolution
(e.g. 300 dpi) agrees with what the PDF was claiming. In an ideal world they would always
be consistent already, but programmers screw this up all the time. That's the thing you
feed to Tesseract (assuming you don't want to do any additional cleaning or something.)
This workflow is kind of sophisticated and maybe not easy for everyone. But it makes
more sense than potentially rescaling the images by rendering to a different dpi.
I just want to put this here because there are several different references being cited in this
bug report about work flow. Please  consider this one authoritative.
It does not however address the core question about dots, which seems like a legitimate
concern. This will be an interesting test document for future development.

@mlissner It would have been helpful, if you had shared the info about your previous tests for this type of document
http://stackoverflow.com/questions/39110300/how-to-provide-image-to-tesseract-from-memory
https://github.com/mlissner/tesseract-performance-testing

We have seen similar documents taking very long (but still not an hour per page!). Therefore, whether it really is a tesseract issue should be investigated further.
@mlissner in order to increase performance and quality, you have to pre-process the image(s) for tesseract. For your specific case, use leptonica (tesseract already depends on it). Count the connected components, if there are too many, apply your filters. In a real word application where your documents have specific characteristics, you will not be able to avoid heavy pre-processing for tesseract in order to achieve reasonable results.
Look how tesseract uses leptonica and CCs e.g.,
https://github.com/tesseract-ocr/tesseract/search?utf8=✓&q=pixConnComp

@jbreiden

This command renders 100% equivalent images for 2.3MB.

gs -dQUIET -dSAFER -dBATCH -dNOPAUSE -sDEVICE=tiffg4 -r300x300 -o gov2.tiff gov.pdf
This command will upscale the original images. It will make them more than 4 times larger. This is unnecessary because the DPI of the original images inside the pdf is 300X300, although the pdf itself falsely 'claims' that the DPI for these images is 72X72.

I was just encouraging -sDEVICE=tiffg4 over -sDEVICE=tiffgray for known black and white images. You are right, care should be taken to avoid rescaling, and that's the primary reason image extraction is safer than rendering.

@mlissner
You could also look at the preprocessing workflow used by pdf sandwich
https://sourceforge.net/projects/pdfsandwich/

https://github.com/DanBloomberg/leptonica/search?q=noise

Lots of responses here, so let me try to respond to as many as I can.
@amitdo and @jbreiden:
I considered using -sDEVICE=tiffg4 over -sDEVICE=tiffgray, but it's not purely black and white, and like I said, the bigger files don't seem to affect performance. Here's a comparison of a gray part of the original PDF:
tiffg4:

tiffgray:

tiffgray is definitely better for this, and since we're doing millions of files, it seems safer to use this approach than to assume all docs are purely black and white (even though it makes big files).
But setting that aside, it seems like using gs is the wrong approach regardless. Seems like the right approach is to extract the images undisturbed. Seems doable, but I'll have to do some research on this. Is it documented anywhere which image formats Tesseract supports natively? There's one question in StackOverflow that seems to address this, but otherwise I don't see a lot of guidance.
@jbreiden you also say:

If necessary, adjust their header so that their resolution agrees with what the PDF was claiming.

This feels wrong to me. In my experience, PDFs are a terrible source of ground truth. I'd expect the header information in the images to be much more accurate than whatever a PDF was reporting. You've provided a lot of information here already, but can you explain why we'd prefer the PDF data over the image data?
@vidiecan: I'll look into counting connected components. Seems like a great way to solve this, if it performs well enough. Thanks for this suggestion.
@Shreeshrii: I looked at PDF Sandwich, but didn't see anything useful. Do you know the code well enough to point me towards the image conversion part?

In this buggy broken world, do whatever it takes to get the resolution right. I rescind my recommendation to follow the PDF settings.  If you crack open gov.uscourts.ctd.18812.88.0.pdf, you can see that it really does contain black and white images. The telltale is BitsPerComponent 1 and the internal use of CCITTFaxDecode, which only works on black and white. I have no idea what ghostscript is doing with that weird halftone.
<<
/Type /XObject
/Filter [/CCITTFaxDecode]
/Length 60 0 R
/Height 3300
/BitsPerComponent 1
/ColorSpace [/DeviceGray]
/DecodeParms [61 0 R]
/Subtype /Image
/Name /Im1
/Width 2550
>>

http://stackoverflow.com/questions/2641770/extracting-image-from-pdf-with-ccittfaxdecode-filter

If you choose to use morphology to remove the site and undo the further, Leptonica is very strong to for C or C++ programmers. A few erosions and dilations would hopefully would do the trick.

@jbreiden, I sense a smart phone was involved in your last comment...some strange words in there.
Also, do you know the image formats supported by Tesseract?

List is here. Discard PDF and PS because they are write-only, and discard SPIX. This assumes that Leptonica is built with all imaging dependencies, which we do for Debian / Ubuntu & friends.
https://github.com/DanBloomberg/leptonica/blob/master/src/imageio.h#L92

@jbreiden
Jeff, your last two messages look cryptic...
If you have been Abducted by aliens, try give us a sign and will rescue you! :)

.--. .-. --- -..- .. -- .- / -.-. . -. - .- ..- .-. .. / -...
I've made a few in-place edits on the bug to
clarify the wording. Hopefully makes more
sense now.
... . -. -.. / ... .--. .- -.-. . / -- .- .-. .. -. . ...

I've made a few in-place edits on the bug to clarify the wording. Hopefully makes more sense now.

I deleted my previous message just after you made the edits. I thought that you didn't like my little joke...
Clearly, I was wrong!

If you have been abducted by aliens, try give us a sign and will rescue you! :)

<

.--. .-. --- -..- .. -- .- / -.-. . -. - .- ..- .-. .. / -...

... . -. -.. / ... .--. .- -.-. . / -- .- .-. .. -. . ...

Jeff, we are coming, stay calm!
LOL

It's good to know Morse, or maybe just to find an online Morse code translator... :)

Even if you use -sDEVICE=tiffgray, you might want to use -sCompression=lzw.

You might want to use -sCompression=lzw.

I just did some simple timings on this.
The good:

Compressed Tiffs are about 1-2% the size of the uncompressed versions (in a test I just did, uncompressed was 137M while compressed was 1.8M!).
Using compressed files used about 30% of the RAM (92MB instead of 301MB).
LZW is a lossless format, so Tesseract generated identical results.
It took Tesseract about the same amount of time to do either format.

The bad:

It takes about twice as long to generate compressed tiffs from PDFs (though this is only a fraction of the total time doing OCR).

The hmmm:

Making a compressed tiff moves the processing burden from disk (making a big file) to CPU (compressing a big file).

Our bottleneck on our OCR server is CPU, so it's actually preferable for us to generate big files that use less CPU than to make small files that don't. OTOH, RAM is expensive, so we'll probably be switching this out. Thanks for the suggestion!

For generating many 1-page images files, instead of one multi-page tiff file,use -o img-%d.tiff.

My first patch (dated March 28)  in this bug https://github.com/tesseract-ocr/tesseract/issues/233 will reduce RAM use in TIFF. It stops Tesseract from buffering the input file before decompression. The patch should also should make the LZW case equal to the non-LZW case with respect to RAM. Note that I haven't tested on this particular example, so I'm saying "should" rather than "does".

Jeff,
Why are we not commiting your patch from March?
On 4 Oct 2016 9:30 p.m., "jbreiden"<EMAIL_ADDRESS>wrote:

My first patch (dated March 28) in this bug #233
https://github.com/tesseract-ocr/tesseract/issues/233 will reduce RAM
use in TIFF. It stops Tesseract from buffering the input file before
decompression. The patch should also should make the LZW case equal to the
non-LZW case with respect to RAM. Note that I haven't tested on this
particular example, so I'm saying "should" rather than "does".
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/tesseract/issues/431#issuecomment-251488440,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o26v-ubHM_MSiloIl-YAaOzocpMPks5qwqlWgaJpZM4KEcsN
.

The plan was for Ray to commit that patch. However, he has been too busy with the upcoming Tesseract 4.0 and over six months have passed. I think it is okay if someone wants to commit the patch. Please do not commit the second patch, though; that should wait until after the next Leptonica release.

Jeff patch was applied, so closing  this issue. If the issue exists in current code, please create new issues.


3.4.x Buttons not showing Focus texture unless 'Normal' texture is set when using keyboard control

Godot version
3.4
System information
Linux
Issue description
On converting from 3.3 to 3.4 I noticed my menu stopped working. It is a basic texture button based system that can be controlled by mouse or keyboard.
The mouse still works, however keyboard navigation does not work if no 'normal' texture is set.
As per the video below and attached code you can move the mouse and it selects the menu items correctly showing the focus texture, however when using cursor up/down it only shows the focus texture if a normal texture is set, as can be seen in the video (or code) when you press up/down nothing is shown until you get to the bottom three buttons. Prior to 3.3 the menu worked as expected and all menu items showed the focus texture.
Note, this is probably an aside as I don't know if this is a bug/feature/the way it is, but in order to initially get focus I have to now explicity call grab_focus() in code on the first menu item. In 3.3 my menu automatically worked with the keyboard without having to first grab_focus()
https://user-images.githubusercontent.com/12863685/147927784-ec157cd3-4709-4834-99c4-cfe06cd08b78.mp4
Steps to reproduce

Open code
Run
Mouse movement works ok
Cursors do not show the focus item on the top half of the menu

Minimal reproduction project
menu_bug.zip

Related to #47280, it separated hover from focus.

CC @godotengine/gui-nodes

Thanks for taking the time to look, I can confirm from a short trial that setting a Normal texture seems to (temporarily) fix it, so presume this is a bug.
But a question if I may: I created the image using an ImageTexture from the dropdown as below. But when I move out of the screen and back it changes from transparent checkboard to black and displays as black on screen. Is this a rendering/gui problem or something else (i.e. should it be black and if it should, should it immediately show black, and how do I create a transparent ImageTexture from the selection?)