ARROW-17045: [C++] Ignore trailing slashes on files in GCS

The R filesystem bindings expect that we can slap on a trailing slash to file paths and they will be ignored in implementation (see this PR for details). This behavior works well in S3FileSystem, but has odd behavior in GCS:
import pyarrow.fs
from pyarrow.fs import FileSelector
from datetime import timedelta

gcs = pyarrow.fs.GcsFileSystem(
    endpoint_override="localhost:9001",
    scheme="http",
    anonymous=True,
    retry_time_limit=timedelta(seconds=1),
)

gcs.create_dir("py_test")

# Writing to test.txt with and without slash produces a file and a directory!?
with gcs.open_output_stream("py_test/test.txt") as out_stream:
    out_stream.write(b"Hello world!")
with gcs.open_output_stream("py_test/test.txt/") as out_stream:
    out_stream.write(b"Hello world!")
gcs.get_file_info(FileSelector("py_test"))
# [<FileInfo for 'py_test/test.txt': type=FileType.File, size=12>, <FileInfo for 'py_test/test.txt': type=FileType.Directory>]

s3 = pyarrow.fs.S3FileSystem(
    access_key="minioadmin",
    secret_key="minioadmin",
    scheme="http",
    endpoint_override="localhost:9000",
    allow_bucket_creation=True,
    allow_bucket_deletion=True,
)

s3.create_dir("py-test")

# Writing to test.txt with and without slash writes to same file
with s3.open_output_stream("py-test/test.txt") as out_stream:
    out_stream.write(b"Hello world!")
with s3.open_output_stream("py-test/test.txt/") as out_stream:
    out_stream.write(b"Hello world!")
s3.get_file_info(FileSelector("py-test"))
# [<FileInfo for 'py-test/test.txt': type=FileType.File, size=12>]

We couldn't always remove the trailing slash since we want to keep it for directories:
https://github.com/apache/arrow/blob/66b0d2a999c8d817fade056e625e4d3ff92a4fff/cpp/src/arrow/filesystem/gcsfs_test.cc#L642-L645
So I opted for a solution that just drops the slash for OpenOutputStream.

Seems reasonable to me.   Is it possible to test this against a generated dataset to make sure nothing breaks there?

Seems reasonable to me. Is it possible to test this against a generated dataset to make sure nothing breaks there?

Do these tests from R seem sufficient? https://github.com/apache/arrow/blob/81af2912b5e088c83631df4cfd52e6d6df2878dc/r/tests/testthat/helper-filesystems.R#L130-L156
I pulled the changes from this PR into https://github.com/apache/arrow/pull/13542, so the above tests will run in CI there. (I can confirm they pass locally for GCS and S3.)

Just wondering: to what extent is this a corner case that just happens to be in our tests, or is there a practical value in allowing trailing slashes?
Because for example also the LocalFileSystem does not allow trailing slashes in a similar example:
In [51]: from pyarrow.fs import LocalFileSystem

In [52]: local = LocalFileSystem()

In [53]: local.create_dir("py-test")

In [54]: with local.open_output_stream("py-test/test.txt/") as out_stream:
    ...:     out_stream.write(b"Hello world!")
...
IsADirectoryError: [Errno 21] Failed to open local file 'py-test/test.txt/'
Detail: [errno 21] Is a directory

(which is an error that makes sense to me, also Python's open will complain that it is a directory and not a file in the equivalent example)

I agree it seems better to disallow trailing slashes in filenames rather than silently dropping them.

Just wondering: to what extent is this a corner case that just happens to be in our tests, or is there a practical value in allowing trailing slashes?

I'm not sure yet what else relies on that. In R, it's just that we pass fs$path("some/path") to functions like write_parquet(). fs$path() returns a SubTreeFilesystem, which I think is just used as a convenient way to pass a path (as the base path) and a filesystem in one object. But as a side effect of using base_path to transmit the path, we add a trailing slash. Using this method fails for LocalFileSystem though:
library(arrow)
#>
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#>
#>     timestamp

example_data <- arrow_table(x = Array$create(c(1, 2, 3)))

fs <- LocalFileSystem$create()
write_parquet(example_data, fs$path("test.parquet"))
#> Error: IOError: Failed to open local file 'test.parquet/'
#> /Users/willjones/Documents/arrows/arrow/cpp/src/arrow/filesystem/localfs.cc:442  ::arrow::internal::FileOpenWritable(fn, write_only, truncate, append). Detail: [errno 2] No such file or directory

Created on 2022-07-12 by the reprex package (v2.0.1)
cc @nealrichardson in case he has any input.
I will see about changing GCS to reject file paths that end with slashes. Should I leave S3 alone? Or should it also reject them?

Hmm, SubTreeFilesystem is meant to take a subdirectory parameter, you are not expected to use it to pass a filename.

Should I leave S3 alone? Or should it also reject them?

It would be fine with me to remove them, but if you hit too many regressions then no need to sweat over it either :-)

Hmm, SubTreeFilesystem is meant to take a subdirectory parameter, you are not expected to use it to pass a filename.

Historical context: ARROW-10254, which points to https://github.com/apache/arrow/pull/8351#discussion_r501252247
We could have the single-file writers (perhaps the readers too) in R prune a possible trailing slash from filenames, if this is the only source of the issue. That logic is pretty well encapsulated in make_readable_file() and make_output_stream() so it should be feasible to do. Shouldn't be a concern for write_dataset/open_dataset since they point at directories.

Oh no! Those do look related @jorisvandenbossche. Is HDFS not in our usual set of CI tests?


Firefox

Would you please make the add-on compatible with Firefox browser? - Thank you.

I'm currently preparing for a major update, and support for Firefox will be added then. Thank you.


global_compile.dart broken file paths on Windows

There is a problem parsing the URI when running in the windows command line. The file path gets truncated down to just c:
Unhandled exception:
FileSystemException: Cannot open file, path = 'c:' (OS Error: Access is denied.
, errno = 5)
#0      _File.throwIfError (dart:io/file_impl.dart:562)
#1      _File.openSync (dart:io/file_impl.dart:413)
#2      _File.readAsBytesSync (dart:io/file_impl.dart:473)
#3      _File.readAsStringSync (dart:io/file_impl.dart:507)
#4      _loadFile (file:///c:/Users/nshahan/code/dart-sdk/sdk/pkg/dev_compiler/tool/global_compile.dart:267:24)
#5      transitiveFiles (file:///c:/Users/nshahan/code/dart-sdk/sdk/pkg/dev_compiler/tool/global_compile.dart:275:18)
#6      main (file:///c:/Users/nshahan/code/dart-sdk/sdk/pkg/dev_compiler/tool/global_compile.dart:79:3)
#7      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:259)
#8      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:148)�

@nshahan was this fixed in https://github.com/dart-lang/sdk/commit/8f5f1a0fb0b4f169eb264ee6c10fb076b4c05643 ?

I was seeing this issue when running on a windows machine. The branch you mentioned worked for me on the project I was working on but was not thoroughly tested on other platforms and I don't want to merge those changes in. I will cleanup.


Add SchemaUtils class to provide schema related utils.

Add GET, POST, DELETE schema methods.
Add compare schema ignore version method.
Add more APIs for setting default null value.
Add annotation for Nonnull and Nullable

Current coverage is 63.01% (diff: 25.14%)

Merging #503 into master will decrease coverage by 0.17%

@@             master       #503   diff @@
==========================================
  Files           709        710     +1
  Lines         32980      33098   +118
  Methods           0          0
  Messages          0          0
  Branches       4977       4986     +9
==========================================
+ Hits          20842      20857    +15
- Misses        10668      10765    +97
- Partials       1470       1476     +6

Powered by Codecov. Last update 0700d09...3d6a37d

Please address the comments, and add unit tests. LGTM otherwise.


Check gulp (and maybe node) versions in liferay-theme-tasks

This is to avoid things like this one, where a user spent some time dealing with an error he could have fixed easily if we have told him about the problem from the tool.
Some people have also asked to check node version. Especially node 12 when using Themes Toolkit v9, which is known to fail due to the "primordials" error.

Is there a suggested way to upgrade to version 10.x.x ?  I used the gulp update option but that also upgraded my theme from Liferay 7.2 to 7.3, which I did not want to do.
We are using Node v12.x.x and need to upgrade to v10.x.x of the the theme tasks and v4.x.x of Gulp.

I used the gulp update option but that also upgraded my theme from Liferay 7.2 to 7.3, which I did not want to do.

I think that's always going to update to reference the latest version of DXP, but in practice, I believe you could safely revert those changes back to 7.2 and keep using v10.x of the toolkit, because (IIRC) there have been no  significant changes in how themes work in the 7.1 → 7.2 → 7.3 range; it's just the version numbers that have incremented.

Thank you for the response, that makes sense to me, just wanted to make sure we were doing it correctly.  I just changed all the references back to 7.2 after the upgrade and everything seems to be working.


PMS hangs sporadically on startup while opening index

The last thing seen in the debug log is similar to:
[0.39133] Sync(): opening search index

Workaround is to delete PMS' cache files:
rm -rf ~/.cache/pms/<mpd_host>/<mpd_port>

I had another hang just now at Sync(): opening search index. Waited a long while and nothing appears to be happening. Killed the process and tried again, same thing.
Deleted its cache directory, tried again, it worked.
Loaded again, working.

This bug might be fixed in #106. @tremby, care to give it a try?

Hope so! Due to the sporadic nature I can't be sure just yet, but I will report back if I still get these kind of hangs.

Thanks. I'll close the issue. Feel free to re-open if if occurs again.


Fix SIGSEGV on connection refused

Fixes SIGSEGV that occurs when http.Do returns an error, for example connection refused

Hi, do you have a code example when the SIGSEGV occurs, and the line in the backtrace?
I'm asking because this shouldn't happen — eg. when you point the client to an inaccessible URL:
ELASTICSEARCH_URL=http://localhost:1234 go run _examples/main.go
Error getting response: dial tcp [::1]:1234: connect: connection refused

Returning right away after roundtrip would make the error not accessible to the logger, see this line:
https://github.com/elastic/go-elasticsearch/blob/8ed748708d21347462d37f4a7cdd1ca267a28436/estransport/estransport.go#L122

Got the idea, it happened only with EnableResponseBody flag enabled, there's no check that res is not nil before accessing res.Body. Updated the MR.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x6ebac5]

goroutine 1 [running]:
github.com/elastic/go-elasticsearch/v7/estransport.(*Client).Perform(0xc000110540, 0xc000124000, 0xc00009e2ee, 0x1, 0x0)
        /go/pkg/mod/github.com/elastic/go-elasticsearch/v7@v7.1.1/estransport/estransport.go:116 +0x535
github.com/elastic/go-elasticsearch/v7/esapi.InfoRequest.Do(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x81f980, 0xc000110540, ...)
        /go/pkg/mod/github.com/elastic/go-elasticsearch/v7@v7.1.1/esapi/api.info.go:85 +0x3ae

Thanks for the update — that's the real fix, actually. I've merged it into master and the version branches.


picard: 2.2.1 -> 2.2.2

Motivation for this change
Picard update from 2.2.1 to 2.2.2.
I haven't added qtmultimedia to enable to the experimental player.
Also acoustic fingerprinting is currently broken as Picard requires an updated version of chromaprint (this was also the case with 2.2.1). See https://github.com/NixOS/nixpkgs/pull/60137 for the chromaprint update.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @ehmry

Hi, thanks for taking the time to review, that error is related to the experimental built-in player, the warning also exists with the current version:
$ nix-shell -p picard --run picard
Warning: config file 2.2.2.final0 was created by a more recent version of Picard (current is 2.2.1.final0)
W: 07:31:26,001 /nix/store/09zh7ygsg93157r8fdk4jgjp0l81l0mp-picard-2.2.1/lib/python3.7/site-packages/picard/ui/playertoolbar.__init__:98: Internal player: unavailable, cannot import name 'QtMultimedia' from 'PyQt5' (/nix/store/6hd6aqf18cw3gsmmfaypdn6ldd8ygk86-python3.7-PyQt5.sip-4.19.18/lib/python3.7/site-packages/PyQt5/__init__.py)

It is easy enough to get rid of the warning by adding qtmultimedia to the derivation:
index dc1118f47e0..fc49e91d414 100644
--- a/pkgs/applications/audio/picard/default.nix
+++ b/pkgs/applications/audio/picard/default.nix
@@ -13,10 +13,10 @@ in pythonPackages.buildPythonApplication rec {
     sha256 = "1iibkvwpj862wcrl0fmyi6qhcgx4q5ay63yr0zyg0bkqgcka0gpr";
   };

-  nativeBuildInputs = [ gettext qt5.wrapQtAppsHook qt5.qtbase ];
+  nativeBuildInputs = [ gettext qt5.wrapQtAppsHook qt5.qtbase qt5.qtmultimedia ];

   propagatedBuildInputs = with pythonPackages; [
-    pyqt5
+    pyqt5_with_qtmultimedia
     mutagen
     chromaprint
     discid

Then you'll be able to enable the player toolbar, however it doesn't work properly (no-audio, no progress bar). I can't tell if this is an issue with NixOS, or with Picard. Either way, it is explicitly an optional feature (e.g. Debian and Arch Linux both list the dependency on qtmultimedia as optional).
There is some info on the player toolbar here: https://tickets.metabrainz.org/browse/PICARD-1488
At some point I might try debugging the feature to get it to work, but so far it's not even producing anything in the debug logs, so it might be a while...


⭐FEAT: Loader Component for contributors page

Feature Request
A loader component for the contributors page would be cool. Even though the results fetched are pretty quick, it can take up to a maximum of one to two seconds for it to complete.

Proposal

A loader component to be added

I would like to work on this, if the feature is planned on being added to the project

yeah sure! I was planning for something like this too. COOL!!


Update types for old Model Server and removed xgboost test models

This PR includes some cleanup for the xgboost models. Followup PRs will include fixes for the online_trainer and new features for the xgboost model.

@KaiyiLiu1234  Sorry for delayed review. Please check my comment about FeatureGroup. The rest looks good to me.
I will rebase my PR after this PR is merged. I think we can merge this PR first.


When hiding titlebar a few white pixels still show on top

Describe the bug
When hiding the titlebar using OverlappedPresenter.SetBorderAndTitleBar(true, false) a few pixels of white titlebar still shows.
Steps to reproduce the bug

Create a new WinUI app.
in app.xaml root tag, set RequestedTheme="Dark"
in MainWindow.xaml.cs constructor add this code to hide the titlebar

            var hWnd = WinRT.Interop.WindowNative.GetWindowHandle(this);
            Microsoft.UI.WindowId windowId = Microsoft.UI.Win32Interop.GetWindowIdFromWindow(hWnd);
            Microsoft.UI.Windowing.AppWindow appWindow = Microsoft.UI.Windowing.AppWindow.GetFromWindowId(windowId);
            var presenter = appWindow.Presenter as Microsoft.UI.Windowing.OverlappedPresenter;
            presenter.SetBorderAndTitleBar(true, false);

Run the app, and observe the white color on top of the window.
Expected behavior
Entire window is black
Screenshots

NuGet package version
1.1.3
Packaging type
Packaged (MSIX)
Windows version
Windows 11 version 21H2 (22000)
IDE
Visual Studio 2022-preview
Additional context
First reported in WinUIEx: https://github.com/dotMorten/WinUIEx/issues/67

@mevey can you take a look?
cc: @btueffers

pong @rkarman

I've repro-ed this bug and filed it task.ms/40617153

Hi @dotMorten, what is the intended goal of hiding the title bar? To draw your own? To have a PIP experience? To have a window that is not movable or resizable perhaps? This is helpful as we try to figure out how to solve this problem.

@mevey I'd ask the Windows App SDK team who added that API  :-)
But a few things come to mind, like splash screens or more custom drawn windows.

Perhaps @ArchieCoder can add details, since he originally logged this bug in WinUIEx

Here is my original issue in WinUIEx: https://github.com/dotMorten/WinUIEx/issues/67
Someone has the workaround: https://github.com/dotMorten/WinUIEx/issues/67#issuecomment-1206649424

@ArchieCoder thanks but I think they are looking for the ‘why’ you were doing this to justify fixing their API

The original goal was to hide the color of the title bar because it didn't find the rest of the app. If the title bar color can be changed, this is then a non-issue anymore.

It seems unchanged in WAS 1.3, so should I consider it a normal behavior? At least, 'ExtendsContentIntoTitleBar' helps eliminate the white space.

we would like to get rid of this little white border as well. we want to have full control over the appearance of the window for our app and this prevents us from doing that

@stevenbrix That's the caption area peeking out there; here's a workaround for now:
var windowHandle = WinRT.Interop.WindowNative.GetWindowHandle(this);
int colorBlack = 0x00_00_00_00;
unsafe
{
    DwmSetWindowAttribute(new HWND(windowHandle),
        DWMWINDOWATTRIBUTE.DWMWA_CAPTION_COLOR, &colorBlack, sizeof(int));
}

Notes:

Congrats, you've now signed up to monitor for DWM theme/colorization, high contrast, etc. changes and set this caption color accordingly! This involves a number of events and window messages. Some documented, some not.
You'll want to bind that to the user's color preferences.
Solid colors only. No fancy art up there or the bar will "reappear".

I am aware of this issue. Seems like something Windows shell puts and unless it gets patched, it cannot be removed. I suggest not using this api, but rolling out a custom titlebar solution using a custom WndProc and using WS_EX_LAYERED and WS_OVERLAPPED styles, but using 1.4 newly introduced InputNonClientPointerSource apis for doing the heavy lifting around Non client message handling. Let me know if any of you need a sample, I can put together a solution you can refer

@pratikone Are you saying OverlappedPresenter.SetBorderAndTitleBar(..., false) is forever broken and that developers will have to handle NC painting/messages manually for no-titlebar scenarios?

I am not saying that at all. I am saying that while this issue gets fixed, there exists a workaround which is much simpler than earlier i.e. let InputNonClientPointerSource apis handle NC messages for a vanilla custom solution.


Nextcloud integration decision 1

How should Nextcloud be integrated in regards to authentication and authorization?
Where should user data be stored?

Why is this in review? Imho, there are several things still to be investigated?


Error when Android Skype client user sends attachment to Skype Bot

While using Skype for Android and sending an attachment (PNG image) to a Node.js Skype bot an error occurs in the bot.
Using latest Skype for Android with Android 6.0.
Error: Incoming webhooks request was invalid, errors: IncomingAttachment.name must not be empty or whitespaces only, request: [{"type":"Image","id":"0-cus-d1-6a58a41c1d14529ebc39bc297065f236","name":"","views":[{"viewId":"original"}],"activity":"attachment","from":"8:bostondv","to":"28:ad3e98f4-848b-4879-b4cc-f88d8d14c2a9","time":"2016-04-16T12:17:28.197Z"}]
    at fromError (/Users/bostondv/Dev/pamfax/skype-bot/node_modules/skype-sdk/lib/messaging/messaging-service-request-processor-v2.js:74:47)
    at Object.processRequestData (/Users/bostondv/Dev/pamfax/skype-bot/node_modules/skype-sdk/lib/messaging/messaging-service-request-processor-v2.js:117:24)
    at MessagingService.processRequest (/Users/bostondv/Dev/pamfax/skype-bot/node_modules/skype-sdk/lib/messaging/messaging-service.js:181:40)
    at BotService.processMessagingRequest (/Users/bostondv/Dev/pamfax/skype-bot/node_modules/skype-sdk/lib/bot-service.js:250:38)
    at IncomingMessage.<anonymous> (/Users/bostondv/Dev/pamfax/skype-bot/node_modules/skype-sdk/lib/utils/messaging-handler.js:33:28)
    at emitNone (events.js:80:13)
    at IncomingMessage.emit (events.js:179:7)
    at endReadableNT (_stream_readable.js:913:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)

It looks like this is on the Skype side of things and unlikely to have anything to do with the node part of your bot. It looks like the attachment name is missing or something. In any case you'll want to report the bug to Skype from their developer dashboard.

Thanks I'll report it to them!

Reopen the issue if you need to followup.

FYI, here's the ticket posted to Skype: https://connect.microsoft.com/UCDEV/feedback/details/2605057/error-sending-attachment-to-skype-bot-using-skype-for-android


AzurePowerShell with Az 3.8.0 causing false positives in Pester

Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name: AzurePowerShell
Environment

Server - Azure Pipelines

If using Azure Pipelines, provide the account name, team project name, build definition name/build number:

Account Name: MerlinPlatform.
Team Project Name: MicroServices
Release Number: releaseId=1709 (latest, but multiple in description below)

Agent - Hosted:

If using Hosted agent, provide agent queue name: Azure Pipelines

Issue Description
Since the Azure PowerShell step has moved to using Az 3.8.0, the step importing Az.Storage 1.14.0 will result in verbose output regarding types (releaseId=1634&environmentId=2445)
2020-05-14T18:51:58.6234067Z VERBOSE: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more
2020-05-14T18:51:58.6242363Z information.
2020-05-14T18:51:58.6623447Z VERBOSE: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more
2020-05-14T18:51:58.6626710Z information.
2020-05-14T18:51:58.8058757Z VERBOSE: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more
2020-05-14T18:51:58.8069290Z information.
2020-05-14T18:51:58.8374865Z VERBOSE: Loading module from path
2020-05-14T18:51:58.8381507Z 'C:\Modules\az_3.8.0\Az.Storage\1.14.0\Microsoft.Azure.PowerShell.Cmdlets.Storage.Management.dll'.

This results in the $Error variable being populated (as caught in pester in releaseId=1634&environmentId=2445)
When retrieving $error[0].Exception.GetBaseException().LoaderExceptions (releaseId=1635&_a=release-environment-logs&environmentId=2447) this contains :
2020-05-14T18:51:59.7716334Z Could not load file or assembly 'System.Threading.Tasks.Extensions, Version=<IP_ADDRESS>, Culture=neutral,
2020-05-14T18:51:59.7717450Z PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.
2020-05-14T18:51:59.7748211Z Could not load file or assembly 'System.Threading.Tasks.Extensions, Version=<IP_ADDRESS>, Culture=neutral,
2020-05-14T18:51:59.7749480Z PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.
2020-05-14T18:51:59.7886716Z Could not load file or assembly 'System.Threading.Tasks.Extensions, Version=<IP_ADDRESS>, Culture=neutral,
2020-05-14T18:51:59.7887836Z PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.

Whilst this wouldn't break a script normally, anything in Pester handling $Error counts, or invoking a Pester script with -EnableExit results  in the script erroring, despite all defined tests passing:
2020-05-14T18:52:02.6143194Z  [-] Error occurred in Describe block 28ms
2020-05-14T18:52:02.6878298Z    RuntimeException: ScriptHalted
2020-05-14T18:52:02.6894185Z    at <ScriptBlock>, D:\a\r1\a\_Infrastructure\tests\storageaccount.tests.ps1: line 166
2020-05-14T18:52:02.7429285Z Tests completed in 4.73s
2020-05-14T18:52:02.7567546Z Passed: 36 Failed: 1 Skipped: 0 Pending: 0 Inconclusive: 0

Setting the Azure PowerShell step from Azure PowerShell version 'latest' to '3.5.0' (the version used the last time this step worked) allows the test script to complete successfully

This is now occurring in other steps.
2020-06-05T09:25:33.2460772Z ##[debug]Loading module from path 'C:\Modules\az_3.8.0\Az.Accounts\1.8.0\Az.Accounts.psm1'.
2020-06-05T09:25:33.4141529Z ##[debug]Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
2020-06-05T09:25:33.4282515Z ##[debug]Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
2020-06-05T09:25:33.7712770Z ##[debug]Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
2020-06-05T09:25:33.7908928Z ##[debug]Loading module from path 'C:\Modules\az_3.8.0\Az.Accounts\1.8.0\Microsoft.Azure.PowerShell.Cmdlets.Accounts.dll'.

Switching to 3.5.0 doesn't resolve it in this case, I've had to switch our pester tests to ignore errors in $error[0].Exception.GetBaseException().LoaderExceptions

This is an issue when importing the Azure PowerShell modules. Ignoring this error in pester should work.


HP Borrowing and Scaling

How does HP borrowing work with scaling up a submission?

SWG Notes:
Question: during parameter borrowing, can a submitter re-submit on a larger scale? If not, can a submitter submit a non-converging model on a large system in the hopes of borrowing HPs?
Proposal:
To resubmit benchmark B with a larger scale X:

You must have submitted some benchmark C at scale Y where Y>=X -- "Prove you can go that big"
You must have submitted B at a technically comparable so that the only modification in resubmission is HP borrowing
The scale X must be larger than anything else you submitted for benchmark B
You cannot withdraw the benchmark C at scale Y and it must be compliant or made compliant
All submissions must have "converged" (N-1 convergences using olympic scoring)

AI: Review this.

Will revisit for v0.8

@bitfort @petermattson  If we are deferring to 0.8, then what did we decide to do for 0.7?  I am unclear on the proposed rule or process.


Wrong interpretation of readOnly and writeOnli concepts in Swagger

In Swagger specification, readOnly and writeOnly refer to whether the annotated property is not present when the DTO is used as a parameter (readOnly) or as a server response (writeOnly). See  https://swagger.io/docs/specification/data-models/data-types/
Swashbuckle incorrectly computes these attributes based on the presence of getters and setters for the property in the C# class.
This causes clients (such as Swagger-UI) to generate incorrect calls, for example when serialising request parameter objects.
The correct approach should be to not derive these properties from readability or writability of the properties in the C# class, but to rely on some custom annotation.
see:
https://github.com/domaindrivendev/Swashbuckle.AspNetCore/blob/ad91d72078e1e2276824b38e2483c1d8ab6e5376/src/Swashbuckle.AspNetCore.SwaggerGen/Newtonsoft/NewtonsoftApiModelResolver.cs#L166

I disagree! Swashbuckle is computing these flags according to "serializer behavior". For example, Newtonsoft will de-serialize JSON payload properties into a corresponding C# property if:

The C# property has a public setter
OR the declaring type's constructor accepts a parameter with the same name

This is the "serializer behavior" and therefore your application's behavior - i.e. specifically what Swashbuckle is describing. If your DTO has properties that don't meet either of the above requirements, then the value WILL NOT be populated into the C# instance and therefore CANNOT ever be written via your API. Therefore, setting readOnly is the correct interpretation.

You mentioned that the current behavior is causing the swagger-ui to generate incorrect calls. Could you provide a concrete example of this? It might help me understand your issue a little better.


Small update of example README.md

update code highlighting: text -> bash
add composer link

Coverage remained the same at 94.872% when pulling 31c89df7df84c3d453e9880fb940591dfcb6ffae on Rotzbua:patch-1 into 6e1d77689f81ce1e383640ea1eafc1847d86cac9 on slimphp:4.x.

Coverage remained the same at 94.872% when pulling 54e356e5dd473675612a21fcbf896b23d7d33d1e on Rotzbua:patch-1 into 6e1d77689f81ce1e383640ea1eafc1847d86cac9 on slimphp:4.x.

Thanks.


animated Class affects overflow-y?

I have a big content on my site, a post. Anyway i had animated fadeIn class on the parent element. At some point of the content you see this:
https://media.discordapp.net/attachments/467293737530425346/616734462662934528/Screenshot_20190829-224214.png?width=169&height=301
It cuts the content off. (There should be more content under the highlighted code)
Seems like overflow is affecting the parent div at some height, it works well if the class animated fadeIn its removed.
Only happens on Chrome and on mobile

Most of the animations can cause elements to overflow as they are literally moved across the screen. How to handle this is up to the developer using the library. This is just how CSS works.
Make sure to use the overflow property when needed to avoid such situation.
Closing this as it's not a issue with the library.


chore: remove fork db snapshot

this fixes an inconsistency highlited by #6463
fork db snapshots were always kept on revert, but state snapshots were always removed, the intended behaviour (ganache, hardhat compat) is to remove a snapshot on evm_revert
also related
https://github.com/foundry-rs/foundry/pull/6366

but this will break existing test suites which now depend on this behavior, aave's included.

forge is unaffected by this.
this merely fixes a bug in anvil


Forward embla up in the hierarchy using React

First of all, congratulations on developing Embla, It's a great library!
I'm trying to interact programmatically with Embla from a component that it's higher in a React component hierarchy, but I cannot figure how to do it. I have a <Container> component that renders another component with the carousel (<EmblaCarousel>), but I want to use a hook in <Container> (potentially useEffect), that access the carousel and calls its scrollTo(). I cannot nest the <Container> in EmblaCarousel>. Is there a way to do this? I tried using React.forwardRef in <EmblaCarousel>, but I couldn't figure how to use it to achieve what I have in mind.
Thanks in advance!

Hello Ian (@ianmethyst)! Thanks a lot. That's encouraging to hear.
Which version of Embla Carousel are you using? Are you using version 4 or later where Embla is initialized with a ref, like described in the documentation?
Kindly,
David

I'm using v4+ as shown in the documentation. I want to know if emblaApi, the second element returned by useEmblaCarousel() can be accessed from outside that component somehow.
As an off-topic, is there a way to disable dragging and re enable it without recreating the carousel? (if it is not possible, I can open another issue as a feature request)

Hello again Ian (@ianmethyst),
Thank you for the additional details. I've setup a CodeSandbox that demonstrates how to forward the ref, in order to access the emblaApi higher up in the hierarchy.
Let me know if it helps and if this is what you had in mind.
Best,
David

Hello Ian (@ianmethyst),
Have you had the change to try the CodeSandbox I created and mentioned here yet?
Kindly,
David

Thank you for your time, and sorry for the late response! Yes, this solved my problem, I didn't realize I could just move the hook up and use React.forwardRef this way
The use case for the off topic question is that I have a carousel that have some items that are zoomable and draggable, and I'd like to disable the drag event in the carousel when an image is zoomed and enable it again when the zoom is reset. I thought I could use reinit, but I was wondering if this would have some kind of performance impact. Sorry if this is a silly question and using reInit is the way to go, I didn't have the time to test it out

No worries Ian (@ianmethyst),
Thanks for the response. I wast just curious if it helped. And I'm glad to hear that it helped, so I'm closing this issue as resolved 👍. I pretty much followed the example in the official documentation page for React.forwardRef.
About the off-topic request. I don't think your app will suffer a performance hit unless you're calling embla.reInit() repeatedly, like 100 times a second or similar. You're welcome to create a feature request for this, although I can't promise that I will choose to add it to the Embla Carousel core.
Kindly,
David


Command of running extract uncertainty

Just a minor thing
For me at least the command only works by removing the =
ns-uncertainty --load-config {PATH_TO_CONFIG} --output-path {PATH_TO_UNCERTAINTY} --lod 8
Maybe I also got it wrong
I appreciate some feedback Thank you

Hi!
Thanks for the feedback. I'm not sure about this, I think it should work both with and without equal signs when you are using -- arguments.  In my case it works with both. It might be a python version or bash script dependency difference. For consistency let's stick with the version with no equals. I will update the README file, thanks.


sick_safetyscanners2: 1.0.0-2 in 'foxy/distribution.yaml' [bloom]

Increasing version of package(s) in repository sick_safetyscanners2 to 1.0.0-2:

upstream repository: https://github.com/SICKAG/sick_safetyscanners2.git
release repository: https://github.com/SICKAG/sick_safetyscanners2-release.git
distro file: foxy/distribution.yaml
bloom version: 0.10.0
previous version for package: null

sick_safetyscanners2
* Initial Release

Release depending on merge of newly released dependencies:
PR https://github.com/ros/rosdistro/pull/27951
PR https://github.com/ros/rosdistro/pull/27950

Release depending on merge of newly released dependencies:
PR https://github.com/ros/rosdistro/pull/27951
PR https://github.com/ros/rosdistro/pull/27950

Can you please rebase this one?  It has conflicts since the dependencies have now been added.

Can you please rebase this one?  It has conflicts since the dependencies have now been added.

Rebased on current ros:master

Rebased on current ros:master


Typos are bad :)

Thanks! 👍

Thanks to you Eloy! I'm fascinated by the shit-ton amount of awesome code you've OS in Artsy. Congrats!!

Glad it’s helpful!

Typo fixes are the best fixes


Added Kernel version

Added the kernel version, currently only on linux

Thanks for your contribution :smiley:
You should add an entry in the modules array in config.json
{
    name:"uname",
    interval:100000
}

Yeah sorry it was late and I wanted to code something :D anyways I fixed it :)

Haha no problem :P


JSON templates for Diagnostic setting for Azure Storage do not work

[Enter feedback here]
Scope parameter is not correct in the templates, it returns errors during deployments
for example
"scope": "[format('Microsoft.Storage/storageAccounts/{0}/blobServices/{1}', split(parameters('storageAccountName'), '/')[0], split(parameters('storageAccountName'), '/')[1])]
Doesn't make sense, because the storage account name cannot use any special charactes, so if the storage account name is mystorage001 , the resulting scope here would be
Microsoft.Storage/storageAccounts/mystorage001 /blobServices/mystorage001 - not correct as it has to be
Microsoft.Storage/storageAccounts/mystorage001 /blobServices/default
The same applies for all other sub services (blob, file, table, queue - for queue, there is not scope mentioned at all)

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 07fc8414-3bff-03f8-fdca-f040eed71389
Version Independent ID: 8196bd5d-b265-fd62-59c4-2a97b34ebf69
Content: Resource Manager template samples for diagnostic settings - Azure Monitor
Content Source: articles/azure-monitor/essentials/resource-manager-diagnostic-settings.md
Service: azure-monitor
Sub-service: essentials
GitHub Login: @bwren
Microsoft Alias: bwren

@TomSzaky
Thanks for your feedback! We will investigate and update as appropriate.

@TomSzaky, thank you for the feedback.
@bwren, could you please review the issue and share your insights. Thank you

Tracking internally
#please-close


SPP 10277 UAT failures item non response

Synopsis
With the recent release candidate UAT testing for totals and components processing was impaired with scenarios where the component list included some or all empty values.
Checklist

[X] Documentation created/updated
[X] Tests created/updated

Description
Selective Editing and Date Adjustment use Pandas NaN to imply empty cell. Totals and Components and Thousand Pounds processing has been tweak slightly to work in a similar way.
Specifically:
thousand_pounds.py
clean_target_variables() - where a string of "NaN" is received or returned, treat as float('Nan')
Return a string representation of numbers, but if the value is None or Nan then do not convert to string
example.py
The csv example is updated to ensure empty cells are treated as nan do bring inline with pandas processing
totals_and_components.py
clean_components_list() - where a string of "NaN" is received or returned, treat as float('Nan')
pandas_example.py
run_all_csvs() - allow the processing of all csvs in a given directory.  Any file that contains "output" in the name is ignored when finding a list of input data files.  This function allows all UAT files provided by MQD to be run.
pandas_wrapper.py
When a value in the input data is None treat as float('Nan')
example_test_data.csv
Add thousand pounds examples where empty cells are present for target_variables
example_test_data.csv
Add totals and components data to test empty cells in component list
Additional CSV files are added for totals and components and thousand pounds processing - these are the UAT files that are run by MQD

When the pandas_example.py runs all the UAT tests for TPC & TCC it produces around 51 output files and these aren't being ignored, unless we want these output files in the repo now. As we are outputting a lot of files now, we can remove lines 10 to 17 in the gitignore file and replace them with _output.csv, so it ignores anything that ends with output.csv =)
If going with the above solution to ignore the output files, we also need to update the TPC example.py line 110 output file name from output_1.csv to output_1.csv.

When the pandas_example.py runs all the UAT tests for TPC & TCC it produces around 51 output files and these aren't being ignored, unless we want these output files in the repo now. As we are outputting a lot of files now, we can remove lines 10 to 17 in the gitignore file and replace them with *output.csv*, so it ignores anything that ends with output.csv =)
If going with the above solution to ignore the output files, we also need to update the TPC example.py line 110 output file name from output1.csv to output.csv.

Good point - update gitignore and changed the name of the output file.


Set custom HTML

Allows setting a custom HTML document and URL without issuing an HTTP request
Resources (Javascript/CSS) are fetched correctly after loading the HTML
Includes specs

General information about my pull requests
As I decided to use capybara and capybara-webkit for screen scraping tasks, I added some new procedures to the webkit_server and Capybara::WebKit::Browser that provide an interface to some of the underlying QtWebKit's advanced functionality.
A possible use case for this is that one can use a fast Javascript-unaware scraper like Mechanize or Net::HTTP and upgrade to a Javascript-aware browser-session using capybara-webkit only as long as dynamic content has to be accessed.
Pull requests that belong to this group of changes:

Advanced cookie management
Set custom HTML

Note: Because I had to add several C++ source files and headers to webkit_server, I decided to give the webkit_server.pro a nicer and more version control friendly layout. This is of course optional, I don't know whether this file is somehow auto-generated?

I think the use case you've described is beyond the scope of what we'd like to maintain. If more people run into a need for this perhaps I'll reconsider, but for the time being I'm going to close this.

@halogenandtoast: I see.
If somebody is interested in this, I opened a fork of the server at https://github.com/niklasb/webkit-server. Also includes Python bindings.


Make Rbac iterable

This could mean:

Changing the current Rbac to support role holder enumeration. This would be a state-breaking change.
Adding an EnumerableRbac alternative that supports enumeration.

An EnumerableRbac could be implemented in two ways:

Implementing the whole iterable collection manually
Just serializing all of the AccountIds into a single slot. Though this is usually frowned upon, in this case it might not actually be the worst idea: The collection has to be iterable, and a smart contract should rarely, if ever, iterate over a collection that is very large, due to gas concerns. This would require some testing.


running app outside application folder crashes app

hi
search path for dlls is searching starting in the currect working directory, supose it needs to use application folder instead.
the chromium dlls fail to be found currently
thanks and regards

@jmecosta , I am not sure I fully understand your scenario. If you want to deploy a Neutronium application, you need to copy every file from the bin directory, including the cef folder and its subfolder that contains cef binary files

so, i get this exception when running outside the folder that contains my .exe
Application: BuildOnlyApp.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: Chromium.CfxException
at Chromium.CfxApi.FindLibraries(System.String ByRef, System.String ByRef)
at Chromium.CfxApi.Load()
at Chromium.CfxApi.Probe()
at Chromium.CfxRuntime.ExecuteProcess(Chromium.CfxApp)
at Chromium.CfxRuntime.ExecuteProcess()
at Chromium.WebBrowser.BrowserProcess.Initialize()
at Chromium.WebBrowser.ChromiumWebBrowser.Initialize()
at Neutronium.WebBrowserEngine.ChromiumFx.Session.ChromiumFxSession..ctor(System.Action1<Chromium.CfxSettings>)    at Neutronium.WebBrowserEngine.ChromiumFx.Session.ChromiumFxSession.GetSession(System.Action1<Chromium.CfxSettings>)
at Neutronium.WebBrowserEngine.ChromiumFx.ChromiumFXWPFWebWindowFactory..ctor(System.Action`1<Chromium.CfxSettings>)
at Neutronium.WebBrowserEngine.ChromiumFx.ChromiumFxWebBrowserApp.GetWindowFactory()
at Neutronium.WPF.HTMLApp.OnStartup(System.Windows.StartupEventArgs)
at System.Windows.Application.<.ctor>b__1_0(System.Object)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
at System.Windows.Threading.DispatcherOperation.InvokeImpl()
at System.Windows.Threading.DispatcherOperation.InvokeInSecurityContext(System.Object)
at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
at MS.Internal.CulturePreservingExecutionContext.Run(MS.Internal.CulturePreservingExecutionContext, System.Threading.ContextCallback, System.Object)
at System.Windows.Threading.DispatcherOperation.Invoke()
at System.Windows.Threading.Dispatcher.ProcessQueue()
at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
at MS.Win32.HwndWrapper.WndProc(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(System.Object)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(System.Windows.Threading.DispatcherPriority, System.TimeSpan, System.Delegate, System.Object, Int32)
at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr, Int32, IntPtr, IntPtr)
at MS.Win32.UnsafeNativeMethods.DispatchMessage(System.Windows.Interop.MSG ByRef)
at System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame)
at System.Windows.Threading.Dispatcher.PushFrame(System.Windows.Threading.DispatcherFrame)
at System.Windows.Application.RunDispatcher(System.Object)
at System.Windows.Application.RunInternal(System.Windows.Window)
at System.Windows.Application.Run(System.Windows.Window)
at BuildOnlyApp.App.Main()
We have a wrapper that calls the Neutronium application, it does not set the working path the the place where the Neutronium application is. So it crashed, i ve fixed from my side by setting the working dir to there.
But i think this use case should be supported out of the box in every scenario.

So if I understand, you are calling Neutronium application from another application(same process). But at the same time, the two processes are located in different folders correct?
Could you try the following fix in
ChromiumFXSession.cs:
line 38:
var codeBase = this.GetType().Assembly.CodeBase;
var uri = new UriBuilder(codeBase);
string directory= System.IO.Path.GetDirectoryName( Uri.UnescapeDataString(uri.Path));
settings.LocalesDirPath= System.IO.Path.Combine (directory, @"cef\Resources\locales");
settings.ResourcesDirPath= System.IO.Path.Combine (directory, @"cef\Resources");
settings.BrowserSubprocessPath= System.IO.Path.Combine(directory, "ChromiumFXRenderProcess.exe");

?

weird, in principle that would solve it. but it does not :(
this is the way i call the application:

then i still get this exception:
Description: The process was terminated due to an unhandled exception.
Exception Info: Chromium.CfxException
   at Chromium.CfxApi.FindLibraries(System.String ByRef, System.String ByRef)
   at Chromium.CfxApi.Load()
   at Chromium.CfxApi.Probe()
   at Chromium.CfxRuntime.ExecuteProcess(Chromium.CfxApp)
   at Chromium.CfxRuntime.ExecuteProcess()
   at Chromium.WebBrowser.BrowserProcess.Initialize()
   at Chromium.WebBrowser.ChromiumWebBrowser.Initialize()
   at Neutronium.WebBrowserEngine.ChromiumFx.Session.ChromiumFxSession..ctor(System.Action`1<Chromium.CfxSettings>, System.Action`1<Chromium.Event.CfxOnBeforeCommandLineProcessingEventArgs>)
   at Neutronium.WebBrowserEngine.ChromiumFx.Session.ChromiumFxSession.GetSession(System.Action`1<Chromium.CfxSettings>, System.Action`1<Chromium.Event.CfxOnBeforeCommandLineProcessingEventArgs>)
   at Neutronium.WebBrowserEngine.ChromiumFx.ChromiumFXWPFWebWindowFactory..ctor(System.Action`1<Chromium.CfxSettings>, System.Action`1<Chromium.Event.CfxOnBeforeCommandLineProcessingEventArgs>)
   at Neutronium.WebBrowserEngine.ChromiumFx.ChromiumFxWebBrowserApp.GetWindowFactory()
   at Neutronium.WPF.HTMLApp.OnStartup(System.Windows.StartupEventArgs)
   at System.Windows.Application.<.ctor>b__1_0(System.Object)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
   at System.Windows.Threading.DispatcherOperation.InvokeImpl()
   at System.Windows.Threading.DispatcherOperation.InvokeInSecurityContext(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at MS.Internal.CulturePreservingExecutionContext.Run(MS.Internal.CulturePreservingExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Windows.Threading.DispatcherOperation.Invoke()
   at System.Windows.Threading.Dispatcher.ProcessQueue()
   at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
   at MS.Win32.HwndWrapper.WndProc(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
   at MS.Win32.HwndSubclass.DispatcherCallbackOperation(System.Object)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
   at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(System.Windows.Threading.DispatcherPriority, System.TimeSpan, System.Delegate, System.Object, Int32)
   at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr, Int32, IntPtr, IntPtr)
   at MS.Win32.UnsafeNativeMethods.DispatchMessage(System.Windows.Interop.MSG ByRef)
   at System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame)
   at System.Windows.Threading.Dispatcher.PushFrame(System.Windows.Threading.DispatcherFrame)
   at System.Windows.Application.RunDispatcher(System.Object)
   at System.Windows.Application.RunInternal(System.Windows.Window)
   at System.Windows.Application.Run(System.Windows.Window)
   at BuildAllApp.App.Main()

by the way i notice that some libs are deployed also. this cef_sandbox.lib is 200MB and made my app explode. Ive remove those libs from the package and everything seems to work. are those really needed?

Regarding  cef_sandbox.lib, you are correct, I will remove this file from next version of chromium.

Regarding the inicial issue, did taking-out this file actually solve it?

nop, ive been already packaging my app without the libs. so not sure whats going on there, perhaps i applied the patch incorrectly or something

Could you please check the path of the cef folder in your deploy scenario and compare it to:
´´´´
var codeBase = this.GetType().Assembly.CodeBase;
var uri = new UriBuilder(codeBase);
string directory= System.IO.Path.GetDirectoryName( Uri.UnescapeDataString(uri.Path));
´´´´

settings.LocalesDirPath= System.IO.Path.Combine (directory, @"cef\Resources\locales");
settings.ResourcesDirPath= System.IO.Path.Combine (directory, @"cef\Resources");

this looks exactly what i have:

cef:

i can try wiht the samples to see if it works

no the samples dont work either:

ChromiumFXWPFWebWindowFactory.cs also contains loading path information. ive change that one also, but did not seem to work either

Do you hav ChromiumRederFxProcess .exe in the same folder?

@jmecosta You shoul also alter line 19:
            CfxRuntime.LibCefDirPath = @"cef\Release";

using same logic as other lines...

ok, after changing all those it works. do you want a pul request?

I should perform some code refactoring, so you don't need to open a PR. I should commit the change today at nigth.
Expect a new version by the end of the week-end.

please close when your ready, thanks

The new package vesrion (Neutronium.ChromiumFx 0.3.0) contains this fix.

@David-Desmaisons everything is working fine now, and the size of the package did not change. thanks


util: make_timeline issues on ucob golden phase

Pulls 7/8 and 9/10 split across golden bahamut even though they are the same pull.
Network_20190328.zip
(cc @JLGarber)

Welp. That's because "The limit gauge resets" is one of the "combat is over" flags.  I'll have to rummage through my old logs and see whether there's anything else we can use for splitting encounters similar to Stormblood tier finals. Fortunately, I think even if not, we can just safely remove that particular check.

Alright, after a bit of log trawling, I can't find anything consistent that accompanies a limit reset. The only thing I found was that Kefka appears to send 80000009 as an actor control during his fight transition. Other than that, the only commonality was the music change, and of course we can't use that.
Given that Square probably won't be implementing many checkpoint bosses, and given that the community is fairly set on deliberately resetting between any phases, I think it's safe to just remove that check and do the manual work necessary in the 1% of cases where it would actually be used.

Yeah, I think considering them "the same pull" seems fine for the purposes of -lf.  If somebody went through and did a real pull after a checkpoint, I think the timeline should still work just fine for both make (because of -p) and test purposes.


Physical Activity ROI - Weekly ROI Metrics Report

Physical Activity ROI Requirements: https://nih.app.box.com/file/1646113563720. Physical Activity ROI to be released under Jan Release Schedule.
For preparing for physical activity return of information, we will need create a new weekly metrics report (separate from the Ops Report) for all Return of Information (ROI) activities. The first section of the report will be for Baseline Physical Activity ROI. As we implement new ROIs, such as genetic ancestry in the summer, we will also place these metrics in this report.
Variables and information related to physical activity report flags such as report delivered, report viewed/declined, report trigger are documented in the requirements doc linked above.
@KELSEYDOWLING7 please consult with @bransteitterbr on how to construct basic metrics based on the variables being stored. Example metric that would be needed: Number and percent of opt-in and opt-out out of denominator of those we had a report delivered.

@bransteitterbr  Please let me know what would be a good time to discuss


Not able to copy and modify login page

There is no option to publish views.
I want to change html for admin login page . Please suggest how can I do that without affect core files.

@bhar-prnv

In app/admin/bootstrap.php
app('view')->prependNamespace('admin', resource_path('views/admin/'));`

add your login template file here
resources/views/admin/login.blade.php

HI , in the Admin/Scaffold I'm getting : Error
Method Illuminate\Database\MySqlConnection::isDoctrineAvailable does not exist.
please help

Sadly Doctrine is no longer supported in Laravel 11.X so just downgrade to Laravel 10 and use it !!

HI , in the Admin/Scaffold I'm getting : Error Method Illuminate\Database\MySqlConnection::isDoctrineAvailable does not exist.
please help

Sadly Doctrine is no longer supported in Laravel 11.X so just downgrade to Laravel 10 and use it !!


WebSubmit: Upload_Files element example does results in a 500er (Version 1.1.4)

Hi,
in Invenio 1.1.4 the Example Response Element "Upload_Files" contains example code, which should be cloned and customized according to the documentation.
However, the code produces a 500er server error, since the python os module which is used in the
line
indir = ParamFromFile(os.path.join(curdir,...) and following lines has not been loaded.
Adding
import os
before these lines fixes this.
Regards
Martin

Dear @martinkoehler would you be willing to send in a pull request with the proposed changes?

Dear Sam,
I sent a pull request.
Regards
Martin
Dr. Martin Köhler
Head of library and documentation
Deutsches Elektronen-Synchrotron DESY
Building 01d Room EG.459
phone: +49-40-8998-1814 Notkestr. 85
fax: +49-40-8994-1814 22607 Hamburg
e-mail<EMAIL_ADDRESS>Germany
----- Ursprüngliche Mail -----

Von: "Samuele Kaplun"<EMAIL_ADDRESS>An: "inveniosoftware/invenio"<EMAIL_ADDRESS>CC: "Martin Köhler"<EMAIL_ADDRESS>Gesendet: Freitag, 9. Januar 2015 09:40:25
Betreff: Re: [invenio] WebSubmit: Upload_Files element example does results
in a 500er (Version 1.1.4) (#2610)

Dear @martinkoehler would you be willing to send in a pull request with the
proposed changes?

—
Reply to this email directly or view it on GitHub .

@kaplun Any updates on this?

I replied in the corresponding PR: #2659. tl;dr Nope.


On ternary plot, start the animation from the centre

As it is now, the circles seem to drop from one side. I would like to have them starting at the position 0.33, 0.33, 0.33. That way we get a feeling on how dynamic they are.

Better still, make the starting position the centroid.


chrome can not detect  eddystone

According to Eddystone Protocol Specification , I have done a eddystone-url pcb board, in  Andriod phone ,Physical Web can detect it quickly,but Chrome can not detect it.
and in iPhone ,Physical Web  can detect it occasionally,Chrome (widget) can not detect it .why?Is it because I in China?

Our early prototype apps showed ALL URLs but once it made it into Chrome, we restrict to only HTTPS URLs for security. if your URL is not HTTPS it will be filtered out. Try changing it to https://google.com or https://github.com/ and see if that works.
If that doesn't work, we have heard that the China firewall sometimes blocks our filtering service. That may also be the problem.

Hi, scottjenson. Thanks for your relay. In fact, https://github.com/ is one of URLs that  I have tried, but it didn't work  correctly.
My iPhone's os version is 9.2.1, and Chrome version is 50.0.2661.95. Test step is:
1.Turn on Bluetooth .
2.Go near beacons broadcasting Eddystone-URL sites.
3.Swipe down notification shade to visit ‘Today’
4. I see "scanning for Physical Web objects..." in the widget.
5.After a while, change to"Physical Web is on" in the widget. but not see eddystone beacon.
I think I did the right thing, but why can't detect it ?
I want to know if Chrome (widget) can detect Eddystone-URL sites  in other areas(except for China)?
Thank you!

Hi @scottjenson
I have an estimote beacon with latest firmware with an URL https.
The beacon is recognized

on iOS  (latest Chrome)
on Android :
-- Physical web app (Google + other implementation)

but nothing is working with Chrome (Public, Beta or dev, latest version available). Location enabled, BL too. etc.
Any clues? Thanks!

It could be a bug in our PWS. Please let me know the URL you are using and we'll take a look at it.

With the Physical web screen inside Chrome, I can see now the url  https://github.com/
So what is the role of your PWS?
If I setup an url more generic (with a redirection), your PWD will block it or not?
redirection is useful to avoid to ask someone to change an url/deploy every week.
Thanks !

Take a look here to learn more about our PWS and why we use it.

BTW, it looks like your website doesn't support https properly. If you go to https://lixar.com you get a broken site (no CSS) whereas if you go to https://www.lixar.com you redirect to the http version.
This means that you're not really serving up HTTPS properly which is likely why the PWS is rejecting it.

Thanks a lot for your support @scottjenson . It make totally sense.

Hello Scott,
I have a similar problem that beacons are detected by app but not by the chrome browser in ios. I have a redirection service but all the url endpoints like https://www.gmail.com, https://www.linkedin.com, https://www.bbc.com  are https complaint. My redirection point is still http though. But that is allowed right ?

Any HTTP URL will work if it ends up at a final HTTPS end point. Try putting your URL into our testing page: verfify.physical-web.org and make sure it passes. If it doesn't, it will give you an error.

@scottjenson how will this effect the Eddystone Gatt Configuration service? Should it be allowed to configure a beacon without a https endpoint from a firmware/app perspective?

The config service is a low level service, it will put anything into the beacon, including 'x.y'. This is on purpose as we want people to be able to experiment. However, our configuration web page cf.physical-web.com does try to do a confirmation before calling the GATT service, but it can't catch all cases. Ultimately, it is up to the developer.

Hello Scott,
thanks for your reply. I get this error from verify.physical-web.org.
Result Filtered (Blocked by robots. See: https://developers.google.com/webmasters/control-crawl-index/docs/getting_started). In my case it is a redirect service hence it disallowing crawling:
'User-agent: *\nDisallow: /'
So does this mean such services will never show up on physical web ? Then how does goo.gl work ?
regards
Nilay

It means your website has a robots.txt file that blocks google. We can't show results if your website is asking us not to! Just change the robots.txt file and we'll start showing you results (give it a few hours to move through the caching)

Thanks Scott. Will keep you posted if that works out...

Hello Scott,
could you explain why even https://bbc.com is being shown as a wrong url
with same robots.txt issue ?
I have corrected robots.txt but same problem still exists...that seems not
to be the issue...
regards,
Nilay
On Mon, Oct 3, 2016 at 4:28 PM, Scott Jenson<EMAIL_ADDRESS>wrote:

It means your website has a robots.txt file that blocks google. We can't
show results if your website is asking us not to! Just change the
robots.txt file and we'll start showing you results (give it a few hours to
move through the caching)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/eddystone/issues/147#issuecomment-251120395,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAz9m5oKu0RdlQNtFj-PKRKm2mgLUO1eks5qwRENgaJpZM4Ia7R7
.

The BBC website has a robots.txt file that says "Don't Index/Crawl me" so.... we don't. We have to respect a websites request to not be indexed.

Yes but it does show up on google search does it not ?
Regards
Nilay
On Tuesday, 4 October 2016, Scott Jenson<EMAIL_ADDRESS>wrote:

The BBC website has a robots.txt file that says "Don't Index/Crawl me"
so.... we don't. We have to respect a websites request to not be indexed.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/eddystone/issues/147#issuecomment-251249637,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAz9m4GRnpnBaJgjTIk3EJGCgqqSnPB7ks5qwYZwgaJpZM4Ia7R7
.

I think the bug in physical web is that instead of taking the final
redirected result it is trying to do filtering at top level. It is a bug.
And probably something else exists. On the first day I could see all the
beacon urls and later they disappeared without any other change
Regards
Nilay
On Tuesday, 4 October 2016, nilay saha<EMAIL_ADDRESS>wrote:

Yes but it does show up on google search does it not ?
Regards
Nilay
On Tuesday, 4 October 2016, Scott Jenson<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

The BBC website has a robots.txt file that says "Don't Index/Crawl me"
so.... we don't. We have to respect a websites request to not be indexed.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/eddystone/issues/147#issuecomment-251249637,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAz9m4GRnpnBaJgjTIk3EJGCgqqSnPB7ks5qwYZwgaJpZM4Ia7R7
.

BBC has an HTTP and and HTTPS website, only the HTTPS has the robots.txt, most likely because it isn't ready yet. So that is why BBC (HTTP) still shows up in google searches.
If your site has a robots.txt file that is limiting search engine access, you must remove it in order to show up in the Physical Web scanner.

Thanks Scott. It took one day before urls become visible on iOS
chrome.....pretty long time for cache to be flushed !
On Tuesday, 4 October 2016, Scott Jenson<EMAIL_ADDRESS>wrote:

Closed #147 https://github.com/google/eddystone/issues/147.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/eddystone/issues/147#event-811908370, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAz9m35dXR-DkFR4-n4guUu3fRgG0Kjtks5qwm0JgaJpZM4Ia7R7
.

Hi Scott,
We try to use Eddystone with our web app on https://m.xamoom.com.
Using your verifier we get:
Result Filtered (Empty page metadata).
But this page has a meta description as well as a title set. Are we missing something? Our robots.txt is empty so it allows everything, we also tried sub urls (which we will need in the end) like https://m.xamoom.com/0ana0 but returns the same error and eddystones don't get shown ny chrome. If we write the url of our standard website to an eddystone (https://xamoocom) it worls, but we need to get m.xamoom.com incl. all path there to work.
Any idea why this does not work?
Best regards,
Bruno

First, this is probably better addressed in the Physical Web repo as this is directed to that project. However, I'm happy to help if this is a quick fix.  Here is the type of markup we look for:
<head>
    <meta charset="UTF-8">
    <meta name="description" content="All you wanted to know about the Physical Web">
    <link rel="shortcut icon" href="icon.png">
    <title>All about the Physical Web</title>
</head>

Nothing fancy, just classic markup.
What I noticed however was that when I tried to you your link it redirected, apparently many times. For now, just try pointing directly to your final page and make sure that works. It most likely will. They you can figure out what crazy redirection thing you're doing. I'm expecting that is what's causing the problems.

Just to sum up what we had to do to make it work for us. The problem was that our web app loads this meta information dynamically. (The redirects you saw where due to the fact that you opened the web app on your desktop or notebook and not on your phone. Sorry that missing information. This will redirect you to the desktop website instead of the web app that is dedicated for mobile use).
Since the meta information got loaded dynamically (JS) into the page once opened, the Google Physical Web Bot, which seems to be responsible for building the notification, did not see it,, because it doesn't wait until everything is there. So our solution was to serve this information right at the moment the page get'S transfered to client right in the HTML not add it via JS.
Thanks for your help!


⚠ gorzdrav.org has degraded performance

In 3c07cfb, gorzdrav.org (https://gorzdrav.org) experienced degraded performance:

HTTP code: 200
Response time: 1246 ms

Resolved: gorzdrav.org performance has improved in 14f57b9.


Provide a way to run a test that needs an external dependency

It would be pretty neat if we had a way to have ./miri run work on tests that need external dependencies -- right now, these are a bit annoying to debug (and for tests that call into libc, this is a regression compared to the state before we added support for test dependencies). This should probably be a separate ./miri run-dep or so, and as much as possible it should share the test-dependencies infrastructure with ui_test/compiletest.

I just ran into this yesterday, where I wanted to just run some files as a quick test, and the libc dependency became a problem. So it'd be great to have some sort of solution here.

This should work for files outside of the test folder. It should also not compare the output to any stderr/stdout files, nor write any stderr/stdout files itself. It should ignore magic test comments. It should just run the file and not have any expectations about the exit code.
Using ./miri test kind of works but it is a crude work-around -- there's a lot of junk in the output, one has to put the file into a particular folder, one cannot overwrite the magic comments.

This would also have been really useful to debug https://github.com/rust-lang/futures-rs/issues/2669.
Judging from https://github.com/oli-obk/ui_test/pull/11 being closed, looks like this "just" needs some wiring up in Miri?

Ah yes, I forgot about this one after implementing the ui_test side of things

I won't get to this this year and January will most likely be spent catching up on other things. Will pick it up again after that.

This works great, thanks!
However printing the entire invocation makes this very verbose and makes it easy to miss the actual program output. Maybe this should be hidden, at least by default?

We can just stop emitting that. I thought it may be practical if something goes wrong but I don't see a good way to hide it conditionally. We could make it a flag before the file name I guess?

If something goes wrong I'd probably rather debug ./miri run and cargo miri run than the complicated dep setup.
So yeah for now I'd vote for not emitting that.


"Account Type" no longer meaningful?

With Gratipay 2.0, "Account Type" on the Settings page doesn't mean anything anymore, does it?  This is confusing now that we have old teams and new teams.

Suggest it be removed from the UI.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/24836358-account-type-no-longer-meaningful?utm_campaign=plugin&utm_content=tracker%2F85909&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F85909&utm_medium=issues&utm_source=github).

Created a PR because I could.  https://github.com/gratipay/gratipay.com/pull/3647

!m @mattbk


album creation is broken - 1 album per photo

I tried to upload a bunch of photos and did set album
      makeAlbums: {
        enabled: true
        use: folderNames
      }

which resulted in one album per photo instead of per folder. This is with an installation as of today (2018-10-22) direct from Github with go, running Debian/sid.

I have seen the same thing on MAC OS 10.14.1. It inconsistently works; sometimes it will add all photos to a single album, others it will add half the photos to a album and the rest will create a new album with one photo in it. Other times it will only create a new album with one photo in it.

Same on Ubuntu 18.10

I suspect this is due to the list albums call returning paginated results per this comment https://github.com/googleapis/google-api-go-client/blob/master/photoslibrary/v1/photoslibrary-gen.go#L2393
my guess is that it might be better to use MediaItemService.Search to search for existing albums instead of paginating through the list (which is likely to be too slow).  See https://github.com/googleapis/google-api-go-client/blob/master/photoslibrary/v1/photoslibrary-gen.go#L2881
I dont really know any go otherwise i'd definitely PR this.

@antonmos it looks like you're right, it works fine on new pictures (albums first on the list) but it starts creating multiple albums for old pictures.

Anyway, it should probably store the album info locally after creating it and re use it instead of listing for every picture. At least per directory.

Just started working with this tool and am running into the same problem.  Not a go programmer, but happy to help out with testing if needed.  Thrilled to see a linux compatible batch uploader with the new API!

I've addressed the album issue here -- it will only create one album per directory. Could I ask your help in testing the branch? @adkfan
https://github.com/nmrshll/gphotos-uploader-cli/pull/45

Yes, absolutely!  One catch....I'm completely new to GitHub.  How do I sync your branch with the fix in it to my local machine for testing?

@tonymet Thanks, with the PR it works, both with album creation enabled and disabled! Thanks.

Happy to test as well. I'm seeing the same "one photo per album" problem with the existing release.

@tonymet Thanks for the PR.  In my test, it only solve partial of the issue.
If the number of albums is large (larger than 20), the PR will recreate empty albums corresponding to folders with files all marked as uploaded.
I tried to move the get or create album code block after IsAlreadyUploaded check and right before the FileUpload job creation and only doing once after entering a new folder.   The modification does not create empty albums, but still recreate a duplicated album corresponding to last discontinued folder with files not uploaded.
I tried to trace the code, and found the root cause is at client.AlbumByname() in GetOrCreateAlbumByName() function.  The list of albums inside client.AlbumByName() only return first 20 items.  This actually confirmed the suspect of @antonmos.  The issue is from the paginated results of google API design. The list albums call only return first page with default page size of 20.

@jrywu big help thanks for the feedback i'll see if I can repro and correct the fault.
If you could create a throwaway with > 20 albums and share the creds with me, that would help me repro and fix.

@tonymet , you are welcome.
There are possibily two solutions.
Solution 1: From google photos API documentation, the album Id is "persistent for sessions".  Since we only need album id when upload photos.  We can store the album id upon created in leveldb like what we did for marking photos to be uploaded in leveldb.  Instead of store mtime+hash for uploaded files, we can store the album id for folders.  This should be a easy fix and  most efficient to store everything local for reducing API calls.  In your PR, the local stored album should be replaced with albumId instead.
Solution 2: To fix the pagination bug in AlbumByName of client.go.   This modification should be PR in the under-layer google-photos-api-client-go instead here.
Here is a quick fix to do pagination when next page token is available ( means album numbers > page size).
I am new to golang and this is a dirty implementation without consider errors or duplicated album names, but it works for me.
func (client *Client) AlbumByName(name string) (album *photoslibrary.Album, found bool, err error) {
ctx := context.Background()
c := client.Albums.List().Context(ctx)
listAlbumsResponse, err := c.Do()
if err != nil {
	return nil, false, stacktrace.Propagate(err, "failed listing albums")
}

if listAlbumsResponse.NextPageToken != "" {
	var albumFound *photoslibrary.Album
	albumFound = nil
	walkPages := func(listAlbumsResponse *photoslibrary.ListAlbumsResponse) error {
		for _, album := range listAlbumsResponse.Albums {
			log.Printf("Matching album title: %s to %s", album.Title, name)
			if album.Title == name {
				albumFound = album
				log.Printf("Matched album %s found with id: %s", album.Title, album.Id)
			}
		}
		return nil
	}
	c.Pages(ctx, walkPages)
	if albumFound != nil {
		return albumFound, true, nil
	} else {
		return nil, false, nil
	}
} else {

	for _, album := range listAlbumsResponse.Albums {
		if album.Title == name {
			return album, true, nil
		}
	}
}
return nil, false, nil

}

I didn't see the above fix and did my own, quite similar. Here it is (modification to AlbumByName in client.go, including optimisation by fetching the maximum number of album names in one go, ie 50):
func (client *Client) AlbumByName(name string) (album *photoslibrary.Album, found bool, err error) {
nextToken := ""
for {

	listAlbumsResponse, err := client.Albums.List().PageSize(50).PageToken(nextToken).Do()
	if err != nil {
		 return nil, false, stacktrace.Propagate(err, "failed listing albums")
	}
	for _, album := range listAlbumsResponse.Albums {
		if album.Title == name {
			return album, true, nil
		}
	}
	nextToken = listAlbumsResponse.NextPageToken
	if nextToken == "" {
		break
	}
}
return nil, false, nil
}

I think that albums id should be cached (as mentioned previously).

Hi,
I've been testing albums creation and it works fine. This is the way I've tested it.

I've created a Folder with more than 50 folders inside.
There is an image inside each folder.
I've called gphotos-uploader-cli to upload the Folder.

The result is having the Folder and the inside folders created. So it seems to be working fine.
Can you explain how to reproduce the error?

in my case the issue is that on each of the folders there is multiple images, and 5 albums created under the same name, on the first 4 albums there is only one picture (different), and in the last album there is the rest of the folder containing

Hi @dotanalon
I've found the problem... This CLI works file by file spawning several threads to do uploads.
But GetOrCreateAlbumByName() is inside these threads and the function is not thread safe. Good catch
Now I'm thinking how to solve it. The best approach would be put GetOrCreateAlbumByName() out of the thread, maintaining the upload file process only inside.

@pacoorozco did you see my comment about pagination https://github.com/gphotosuploader/gphotos-uploader-cli/issues/19#issuecomment-450782703 ?

@antonmos Yes I've seen it. Current master branch has a fix for albums pagination, but this doesn't address the problem with album creation...
There are several concurrent processes calling GetOrCreateAlbumByName(). It's happening that when one process is creating the album, another process doesn't know about it, so it will try to create it again. Due to the possibility of creating several albums with the same name, different upload() process are using different albums IDs. You can see it here.
This bug was probably introduced when concurrency was implemented. Previous of that the album pagination was causing some issues, too. :cry:

Hi @dotanalon and @antonmos
Can you test source code on branch issue-19 or if you prefer use binaries for v0.8.1-pre.
It should fix the problem... right?

I will test it in few hours and update..
Thanks,

@dotanalon were you be able to test it?

Not yet...
Sorry, will try to do so tomorrow
Thanks,

i tested it and problems persist.
When i uploaded to an account that had 17 albums it was working fine. New album got created and all the pics went into it.
When I uploaded to an account that had about 190 albums, duplicate albums got created and pictures got split up between them.
Let me know what other info i can provide to help!

@antonmos It's very strange, because the number of albums will not affect how the code is creating it. When you are referring to When I uploaded to an account that had about 17 / 190 albums, It look like you are referring ti the amount of albums that already exists.
Bear in mind, that gphotos-uploader-cli is only able to upload files to albums that has been created by gphotos-uploader-cli. If you are trying to upload pictures to existent albums (created outside this CLI) you are going to end up with a new album with the same name.
Are you trying to upload to existent albums? Existent means not created by gphotos-uploader-cli.
Thanks for taking time to get more information in order to solve the issue.

no, that duplicated album was not already there.
However, I think when i just started uploading the upload stopped due to running into .DS_Store file.  I had to add that to exludedPatterns and restart it.
Is it possible that the db did not get updated with the reference to the album that got created?

@antonmos We are not keeping album's information locally. So It doesn't matter if you retry without cleaning local DB.
So you have a collection of 190 albums that are not already created in Google Photos (or it were created by gphotos-uploader-cli). Then, when you run the google-photos-cli (using v0.8.1-pre)  to upload more pictures, you see new albums are created with duplicated names, right?

So you have a collection of 190 albums that are not already created in Google Photos (or it were created by gphotos-uploader-cli). Then, when you run the google-photos-cli (using v0.8.1-pre) to upload more pictures, you see new albums are created with duplicated names, right?
Yes, but it my case it was interrupted.

Bear in mind, that gphotos-uploader-cli is only able to upload files to albums that has been created by gphotos-uploader-cli. If you are trying to upload pictures to existent albums (created outside this CLI) you are going to end up with a new album with the same name.

Per above, if you are not keeping track of albums locally, then wouldn't restarting the upload after interrupting due to .DS_Store, result in the duplication?

Just so i understand, if you are not keeping track of albums locally, how come photos-uploader-cli is only able to upload files to albums that has been created by gphotos-uploader-cli?

It gets album's list from Google Photos by folder's name
El dom., 8 sept. 2019 18:37, Anton Mostovoy<EMAIL_ADDRESS>escribió:

Just so i understand, if you are not keeping track of albums locally, how
come photos-uploader-cli is only able to upload files to albums that has
been created by gphotos-uploader-cli?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gphotosuploader/gphotos-uploader-cli/issues/19?email_source=notifications&email_token=AASUPQGFK3G4ZAUYWLSAALTQIUS4FA5CNFSM4F6MFU5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6FT5LY#issuecomment-529219247,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AASUPQC2JXHVAYOMVIXDU5TQIUS4FANCNFSM4F6MFU5A
.

I tested it again and was able to reproduce it without interrupting the
upload.
Btw, I am uploading a folder which contains 10+ other folders, which
actually contain the photos.
On Sun, Sep 8, 2019 at 1:07 PM Paco Orozco<EMAIL_ADDRESS>wrote:

It gets album's list from Google Photos by folder's name
El dom., 8 sept. 2019 18:37, Anton Mostovoy<EMAIL_ADDRESS>escribió:

Just so i understand, if you are not keeping track of albums locally, how
come photos-uploader-cli is only able to upload files to albums that has
been created by gphotos-uploader-cli?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/gphotosuploader/gphotos-uploader-cli/issues/19?email_source=notifications&email_token=AASUPQGFK3G4ZAUYWLSAALTQIUS4FA5CNFSM4F6MFU5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6FT5LY#issuecomment-529219247
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/AASUPQC2JXHVAYOMVIXDU5TQIUS4FANCNFSM4F6MFU5A
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gphotosuploader/gphotos-uploader-cli/issues/19?email_source=notifications&email_token=AAAAVGKLKVLA6JDFDJODSHDQIU5NBA5CNFSM4F6MFU5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6FWBJY#issuecomment-529227943,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAAVGKQXRGHLUGOC4HDPT3QIU5NBANCNFSM4F6MFU5A
.

As far as I see the issue seems to relate to the concurrency logic. I have tested with a completely empty photos account - no existing photos, no existing albums. With the default settings and the latest source from master I am able to replicate duplicate albums immediately:
The first three photos being uploaded are all in the same folder. They ended up in 3 separate folders each with the same name. Next 2 are from the next folder; again they end up in duplicates.
I changed const maxNumberOfWorkers = 1 and retested as suggested by @pacoorozco above and results are as expected - no duplicates.
I am now testing with the issue-19 branch. Same configuration and same photos but back to the default concurrency... as of yet with no duplicates. I'll leave it running and see how it gets on.

@mjhatcher, can you confirm that issue-19 branch solves the album duplication?

@pacoorozco, I can’t definitively say yes but it does look good.
It’s been uploading away, has done a few hundred photos across 20+ albums. Has stopped and been resumed a couple of times too. No duplicates yet.


Create Jenkinsfile

Why was this change made?
Moving the build from jenkinsqa to sul-ci

Coverage remained the same at 36.885% when pulling 0bd117631651f63b164a395bcad8776a5ba2c275 on cbeer-patch-1 into e652494562a6bd98e25a5ca4968eeb841b74a8dd on master.


Java JUnit support

Tracking issue for JUnit support.
Experimental support was added in v1.3.0 with java-junit reporter.
Implementation is based on test result files taken from lhotari/pulsar.
Due to lack of my own experience/interest with Java ecosystem no java project to create test fixtures was added. There is also no documentation how to use JUnit to get XML with results - I expect there is plenty of other resources about this topic.
@lhotari
Here you can see how it looks: https://github.com/dorny/pulsar/runs/2052225393?check_suite_focus=true
My setup is based on your lh-refactor-pulsar-ci-with-retries branch.
By purpose I made one test fail to verify failure annotations are working.
I also modified CI workflow to upload merged-test-report.xml instead of XML per test class.
It works both ways but with XML per test class approach you get more noise in the report.
I've also noticed there are some duplicates.
Multiple test results with same name are also present in XML so it's probably not a bug. Still I'm not sure why is that and If I should handle it in some specific way.
Please keep me updated how it works for you.

Wow! Nice work @dorny . This is really helpful. Thank you!
btw. In the Pulsar build, there are multiple reason for the duplicates. Some tests are run multiple times with different configurations. There are also a few cases where the same class name is reused in different modules. There's also currently some test retries in place for handling flaky tests. The solution is currently such that if any of the test retries succeeds, the build will pass. There are plans to change the flaky test handling so that test retries will be enabled only for tests that are categorized as "quarantined".
I'll let you know how the work proceeds. Thank you once again for putting the efforts in supporting java-junit and testing it extensively with the Pulsar build!

Complete beginner in the field so not sure if this is a user problem (i.e. me) or an issue in the test-reporter.
I am using the MATLAB action to generate JUnit xml files via MATLABs test system , and then run test-reporter on the generated files. Here is an example of a generated xml
<?xml version="1.0" encoding="utf-8"?>
<testsuites>
   <testsuite errors="0" failures="0" name="test_operator_abs" skipped="0" tests="2"
              time="0.50334">
      <testcase classname="test_operator_abs" name="test1" time="0.23014"/>
      <testcase classname="test_operator_abs" name="test2" time="0.2732"/>
   </testsuite>
   <testsuite errors="0" failures="0" name="test_operator_alldifferent" skipped="0" tests="2"
              time="0.57926">
      <testcase classname="test_operator_alldifferent" name="test1" time="0.29553"/>
      <testcase classname="test_operator_alldifferent" name="test2" time="0.28373"/>
   </testsuite>
</testsuites>

When test-reporter works on this file, I receive the following
  with:
    name: Junit
    path: *.xml
    reporter: java-junit
    list-suites: all
    list-tests: all
    max-annotations: 10
    fail-on-error: true
    token: ***
Check runs will be created with SHA=f82f3d9235619082511ed24aadc1ab5a73a27e40
Listing all files tracked by git
Found 1783 files tracked by GitHub
Using test report parser 'java-junit'
Creating test report Junit
  Processing test results from operators.xml
Error: Cannot read property 'time' of undefined

Found it: The outer is not expected and causes issues. trimming those and it works.

...although not completely, as only the first suite is added to the report. The second test_alldifferent is lost when i delete the opening and closing testsuites commands which makes it run

Hi! Thanks for reporting the issue 👍
I've only tested the processing of Java JUnit XML against one example so I'm not that surprised about this.
Anyway, it should be quite easy to fix. I will look into it asap.

Great to hear, your package appears to be what can move me from an home-baked system to full automation in github so thanks for your efforts.

@johanlofberg:  The issue was not in presence of <testsuites> - parser already supported it. The time attribute was missing.
JUnit XML is a good example of something that looks like a standard but actually, every implementation is different.
In your case, the XML was created by Matlab instead of Java library. I've modified the parser so it can handle it.
This action has no official support for Matlab. I would have to add tests and some documentation and unfortunately, I don't have time for it now :) Anyway if you find some other issue please report it and I will try to fix it.
Fix is here: #115

Awesome. All 29 tests tests in the suite are now listed and included, but it appears to misclassify the 3 failed tests as they are listed as pass despite having failed and being listed as such in the xml
   <testsuite errors="1" failures="0" name="test_sdpvar_geomean_complex" skipped="0"
              tests="1"
              time="1.9047">
      <testcase classname="test_sdpvar_geomean_complex" name="test1" time="1.9047">
         <error>Error occurred in test_sdpvar_geomean_complex/test1 and it did not run to completion.
    ---------
    Error ID:
    ---------
    'MATLAB:assertion:failed'
    --------------
    Error Details:
    --------------
    Error using test_sdpvar_geomean_complex>test1 (line 12)
    Assertion failed.</error>
      </testcase>
   </testsuite>

https://github.com/yalmip/YALMIP/runs/2579024508?check_suite_focus=true

created xml
junit.zip

I see now that I am raising errors instead of failures so the problem is on my side, as your code only check pass/fail/skip

Well, both errors and failures are valid in JUnit. It just happens that XML I've worked with during implementation had only failures.
I will make it work with errors too. Hopefully this is the last obstacle. I will let you know when it's ready.

Hey, we've recently integrated your action into our workflow and really like the results.
I realised today the issue above where errors get treated like passes is also effecting us.
Let me know if you want some help testing your fix.
Thanks for your efforts!

I've fixed the issue in #118
It should handle the <error> and <failure> the same way.
The v1 branch already points to this version.
Could you please confirm if it works for you now or is there any other issue?

So far so good. From what I can see it seems to be handling the errors now. I'll let you know if I find anything else.
Thanks again.

First; thanks for a great action helping us migrate to GitHub 😃👍
We are using NUnit with its output transformation to get JUnit-style results. But when a test fails I don't get the failure message anywhere in your generated report, just first line/item of the stack trace.
Is there a way, when a message is present, to have the message-attribute show as the error text, and put the complete stack trace under Raw output and Show more?
Test result XML:
<testcase name="SearchStreetsWithFilterNoHit" assertions="0" time="0.008161" status="Failed" classname="MyAppName.Client.Adapters.MapAdapter.MapAddressSearchTest">
  <failure message="System.IO.DirectoryNotFoundException : Could not find language resources for neither given culture 'nb-NO' or fallback 'en-US'.">   at MyAppName.Client.Adapters.MapAdapter.MapNavigation..ctor(IMapAdapter mapAdapter, ApplicationViewModel applicationViewModel) in C:\actions-runner\_work\MyAppName7\MyAppName7\MyAppName.Client\src\Adapters\MapAdapter\MapNavigation.cs:line 337
   at MyAppName.Client.Adapters.MapAdapter.MapBaseTest.Setup() in C:\actions-runner\_work\MyAppName7\MyAppName7\MyAppName.Client\src\Adapters\MapAdapter\MapBaseTest.cs:line 34</failure>
</testcase>

GitHub action report:
❌ SearchStreetsWithFilterNoHit
	at MyAppName.Client.Adapters.MapAdapter.MapNavigation..ctor(IMapAdapter mapAdapter, ApplicationViewModel applicationViewModel) in C:\actions-runner\_work\MyAppName7\MyAppName7\MyAppName.Client\src\Adapters\MapAdapter\MapNavigation.cs:line 337

GitHub code annotation:
Clicking "Raw output" shows the rest of the stack trace:

GitHub action annotation:

Just for fun, I tested with junit file export from pytest
And it actually works
https://github.com/omBratteng/studious-waddle/runs/3535713080

Another person here who wishes we had the following (or a workaround)
Is there a way, when a message is present, to have the message-attribute show as the error text, and put the complete stack trace under Raw output and Show more?


Add NIAK intra-network average connectivity measure

The python code is here: https://github.com/SIMEXP/ad_sz/tree/new_afc_scripts/scripts
We might want to have a flag for users to indicate if they want this measurement to replace diagonal or not.
There should be detailed documentation for this measure as it's not conventional.
@clarkenj can you help with the documentation part?

Yes I can take the documentation part


How to run stateless custom backend api?

Hello, I am trying to run custom backend API in serverless way. So, it is by default stateless. Moreover, containers are scaled up on demand and scaled down to 0 when there is no workload
All works fine except from model versions. There are multiple problems:

Over the time there are a lot of newly created model versions in LabelStudio UI like on the following screenshot:

Ideally, I would like to have single version INITIAL

From time to time there are exceptions like OSError: Result file /data/models/1642448814/job_result.json specified by model_version doesn't exist

Is it possible to run stateless custom backend API?

@kemics As I remember you can return your model version in the predictions of predict() method, e.g.:
def predict(...):
    ...
    predictions = [ { "model_version": "my-only-one", "result": [...], ... }]
    return predictions

@makseq if I add model_version key to prediction then API returns JSON like {"model_version":"INITIAL","results":[{"model_version":"v0.0.1","result":[],"score":0.0}]}. Now there are two keys model_version on different depth. Is it expected?

Hi @kemics
Could you please provide your ML configuration in Label Studio?
Do you have a storage for your containers?

It seems that after you stop them and start over again you are getting new model_version

The problem was with setting Start training after any annotations are submitted or updated in LabelStudio ML configuration. Right now it works as expected
@KonstantinKorotaev thank you for your help with investigation