CLAP support development blog

I am opening this ticket as a sorta blog progress report style for anyone interested on following CLAP support in DPF.
I will be working on this for the next 2 months, starting with the basics for running a few plugins for this month.
We can set dragonfly reverb as a target, as that has parameters, meters, UI and of course audio.
Then next month going deeper in CLAP integration, for having time position, latency, and all other goodies.

Let's start things today.
I have put in place "stub" implementations which do nothing but allow things to build, in preparation for new plugin formats.
I was thinking of doing things in a "clap" branch, but as I am going through the files that will need tweaking I see old code that needs maintenance and little tweaks. I do not want those general tweaks and fixes to be stuck behind a specific branch, so I will just push the clap things directly into "develop".
Also, my initial thought for this development was to do regular live streams accompanying it. But seems better not, I keep waiting for the right moment to do a stream (cant do it when I only have <1h dev time, or when I expect to have regular breaks because I need to do something on the side, or other life things..), this delays the actual time spent working on the feature which is not good.
Other people likely have better discipline, a fixed working routine and other habits that allow to work on such things in a more consistent way. And I don't..
Plus not always in the mood to appear in a stream...
You get the point, I will not do live streams after all.
Let's focus on the technical stuff.
We call today "day 0".
I put in place the stubs and needed files to build them, but there is no clap specific code in the codebase at all yet.
So we can track how long it takes to put a complete implementation in place.
I will try to make report/summaries on the status on a regular basis.

Results of first day:

Get a plugin built and scannable (core spec, incomplete but works)
Define audio ports (non-proper hacky way for now)
Setup audio processing functions

Works 🚀

Core API is quite simple, so fast start is not unexpected.
The dev docs are missing on some critical details and have quite a few typos and grammar mistakes, will open tickets about it soon.
Already started some initial code for UI and parameters, but not pushed yet.
Will report on that next time.

Results of 2nd day:

improve parameter handling
start implementing UI stuff, only macOS/windows working for now (need to hook the event/fd loop stuff)
passing state from UI to DSP is possible
passing parameters from DSP to UI and vice-versa

can start to see how the code needs to be structured for clap support.
quite a few things that clap needs were already done for vst3, so I have been moving a few vst3-specific functions into common code.
still only works for single-bus plugins (those with only a fixed set of ports, which is the majority of dpf plugins).
but that should be enough to get most plugins to work.
so implementation is progressing at a good pace.
I will (attempt to) provide clap-enabled FX and Synth variants of https://github.com/DISTRHO/Cardinal for its 22.09 release, release date TBD.

I am giving up counting by days, there is not always something interesting to show every time.
Would be cool to know how long the implementation takes, but also puts pressure to finalize it quickly which can lead to pressure-inducing mistakes.
So, will just report here whenever there is anything interesting to show.
This time, it is transport information.
Verified to work with bitwig and Cardinal.

Sending notes from UI to DSP also works now, plus some MIDI input related things are in place but it MIDI input doesnt work yet.

We are close to the first target, dragonfly is only missing 1 feature to make it fully functional - state extension.
Hosts can still save parameters without it, but dragonfly uses this to indicate which "preset" is selected on the GUI.
Spent some time on Linux to ensure the UI works there, can confirm it is the case.

Even resize works now (tested both host-initiated and plugin-initiated).
REAPER ignores the "adjust size to fit" thing though, so it can go below the minimum size.

Not a lot of development news recently, as I have been stabilizing the base implementation and doing tests, fixing the odd issues here and there as they appear.
As CLAP is so new, hosts can still do things in an odd way as they pretty much only been tested with a very small pool of plugins. So far it has been quite okay to adjust once a problem is found. Seems mainly Bitwig, where:

I only report CLAP_NOTE_DIALECT_MIDI as supported, but Bitwig sends us fancy notes anyway (which should only be done if plugin reports it as supported)
X11 UIs fail to init if a window is not created by the time Bitwig calls set_parent, so we just create the window during it instead of waiting for the show call

Now that things are getting stable, I already made a few releases with DPF+CLAP in place.
https://kx.studio/News/?action=view&url=introducing-masterme
https://kx.studio/News/?action=view&url=cardinal-2209-released
https://kx.studio/News/?action=view&url=ildaeil-v11-released
For the last one, yeah you can use Ildaeil CLAP as a way to load LV2 😅
Loading x42-plugins (which are LV2 only) inside Ildaeil-FX CLAP in Bitwig:

Starting new month, developments will begin again.
For testing more specific and detailed features I find that doing the host side helps quite a lot. Because you get to see and experience things not just from the plugin side, but from the host side as well.
On DPF-based plugins I typically use Carla to test features and do automated runtime testing, which is not possible until Carla supports CLAP natively.
Well, that is about to change!

Scanning already works, so the next bits are hosting.
Since CLAP has many similar ideas to VST2&3, likely I just need to copy the implementation and tweak a few things. That is what Bitwig did too anyway :P
For the advanced/next features, it is just the more deep plugin things:

latency reporting
multi IO
parameter and port grouping
CV

perhaps parameter modulation? not sure how to best fit into DPF API yet.

More indirect progress, Carla hosting with parameters and MIDI, and basic time and GUI support

This finally allows to try out all the small details for proper CLAP support in DPF.
Thanks to it found a little bug where memory corruption could happen when plugins had 0 parameters (like Ildaeil).
There was another issue with high-dpi screens on macOS + Reaper, spotted by u-he folks.
Already fixed that in git, but this means existing releases of DPF-based plugins are in need of an update.
Will wait 1 more week for more possible fixes, then start doing release fixes.

Some little news. Yesterday was release day, we got new Cardinal and Ildaeil that bring all CLAP fixes so far.
Additionally, due to Ildaeil reusing Carla for hosting, we can now use it to load CLAP plugins in non-CLAP supported hosts!
And vice-versa, of course.
Anklang (which only supports CLAP) loading LV2 plugins:

Ardour (which does not support CLAP) loading CLAP plugins:

Good news, multi-IO is in.
No fancy screenshots for this, it is just technicalities this time.
Did the testing with https://github.com/DISTRHO/ndc-Plugs which includes a DPF port of https://www.niallmoody.com/ndcplugs/ampimposer.htm
This plugin is nothing fancy at all, but allows to test and verify the feature. Also with this I noticed that CLAP multi-IO hosting was broken in Carla, so that is corrected on latest git now.
On this same topic, now that we have multi-IO, I went ahead and enabled the Cardinal main variant for CLAP.
Still does not have CV, but has 4x stereo audio pairs, with only the first being marked as "main".

Oh forgot to mention, with CLAP hosting being available in Carla now, I have began to enable the runtime tests for plugin CI builds, as all other DPF-supported formats were handled already.
Can be seen in for example https://github.com/DISTRHO/ndc-Plugs/actions/runs/3301517161/jobs/5447073018
Need to spread out the CI workflow file to all projects now... but good timing as github has deprecated a bunch of stuff so I need to update these anyway.

With a few more fixes in place, I went through the different kind of plugins in DPF-Plugins collection and verified the behaviour of each type.
And with that done, we get new release https://github.com/DISTRHO/DPF-Plugins/releases/tag/v1.6
15 new CLAP plugins at once! :clap: :clap: :clap:
Fixing ProM also involved fixing some long-standing bugs in DPF when using OpenGL3 mode, among other details.
(a few things were broken on WIndows and I didnt notice, as I rarely use that)
Since with VST3 and CLAP in DPF having some tiny details not yet implemented, I revised DPF's README to no longer say that all plugin implementations are complete. They are mostly complete, for what most devs care about anyway.
Instead the README now links to https://github.com/DISTRHO/DPF/blob/main/FEATURES.md which I have updated to have CLAP as part of the feature table.
I was hoping to do some documentation related work this month, but the more slow times now need my attention for planning/preparations for a move. Going back to Berlin, land of u-he and bitwig among others :sunglasses:
Plus there is also the Ubuntu Summit 2022 next week, another trip in between others :airplane:
In any case, will keep this thread updated for CLAP + DPF related things as usual, even if going slower in the next few weeks.

Hey all, for those potentially interested in DPF I have created a plugin template at https://github.com/DISTRHO/imgui-template-plugin
Uses cmake for build system and ImGui for the GUI, so hopefully some will be familiar to at least one of these.
It is not documentation, but can be helpful for easy experimentation.
I only tested Linux builds for now, so let me know if something is broken.

Oh for those curious about imgui, their project page at https://github.com/ocornut/imgui has quite some info.

I managed to get the DPF+ImGui example plugin running on Windows with CMake and MSVC++ without too much trouble.
I targeted both CLAP and VST3, and both plugins seem fine in REAPER.
I've been developing some really interesting drum programming software for the last 15 years or so... I'm looking to develop a bridge plugin that let's me easily integrate my stuff into traditional DAW environments. Looking at using DPF to get into it.
I'm already using Dear ImGui extensively, so this seems like a pretty solid entrance point.


Add LVM2 Thin Volume / Pool Support

Extends LVM logic to parse LVM2 Thin Pool Metadata and lookup mapped blocks when reading from thin volumes.

@roliveri please review

I had to stop reviewing...will come back to this later

One general comment is that it's typical in Ruby to have a separate file per class.  I know that other SSA classes hadn't done this in the past, so I'm not sure if this was done here to be consistent with that, or if it should change here since it's all new code.  @roliveri I guess that's really up to you.

Also, does this read the entire BTree into memory when used in practice?  If so, that would seem inefficient to me, as the point of the BTree is to avoid that.  I can't tell from the code, but it feels that way (perhaps that's only the entries method)?

@Fryguy pr updated to incorporate feedback. A few things to note

the btree does construct a hash for key / entry pairs and makes this available via the to_h method

the entire btree is not loaded into memory, rather at any given time only the level being referenced / looked up will be. Thus if we traverse the block map tree looking for a particular block, only the path of tree nodes leading to that block will be traversed.

the entire lvm logic was split up into modules in the second patch, perhaps this is a good candidate to refactor into it's own gem (abstracting the internal 'consumption' of disk bits as well as the io interface providing client access)

Let me know if there is anything else

@roliveri please review

Yes, I should complete the review sometime next week.

@movitto It looks good. Just address the dumpMetadata comment, and it should be ready to merge.

@roliveri think that was just a relic left over from debugging, but added the conditional in any case

@roliveri updated


Too many exe files

Unhandled Exception: System.Exception: Too many exe files
   at DotnetMono.Main.getExecutable(String path)
   at DotnetMono.Main.main(String[] argv)

Should assume project name or read AssemblyName
Or maybe figure out how dotnet run is doing it.

Closed via #9


kubectl ket provider doesn't show the kubernetes provider config

For the default crossplane providers you can see the provider configs via
kubectl get provider

The kubernetes provider doesn't show up in that list as of know.
That's because in the CRD the categories section of the providerconfigs.kubernetes.crossplane.io crd is missing:
  names:
    categories:
    - crossplane
    - provider

This probably has to be generated through some kubebuilder annotations.

@turkenh  fixed by https://github.com/crossplane-contrib/provider-kubernetes/pull/22 - you close the issue

seems not yet fixed, I use kubectl get provider/ providerconfig and it hanging

Not fixed here either, no ProviderConfig CRD available when installing Crossplane 0.24.x.


Unable to resolve module error on Windows

Description
I can't compile the app on Windows, althrough it can be normally compiled on MacOS. The dev env on the Windows computer is set up and if I init a new react native project I can normaly compile it. I have tried quite a few things but it looks like it is a bug. Its not connected with the code on the project because if i delet the modules that are cousing the problem, new ones appear.
I would greatly appreciate any help.
React Native version:
System:
OS: Windows 10 10.0.18363
CPU: (8) x64 Intel(R) Core(TM) i7-3740QM CPU @ 2.70GHz
Memory: 4.68 GB / 15.89 GB
Binaries:
Node: 12.18.4 - C:\Program Files\nodejs\node.EXE
Yarn: 1.22.4 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
npm: 6.14.6 - C:\Program Files\nodejs\npm.CMD
Watchman: Not Found
SDKs:
Android SDK:
API Levels: 28, 29, 30
Build Tools: 28.0.3, 29.0.2, 29.0.3, 30.0.2
System Images: android-29 | Android TV Intel x86 Atom, android-29 | Intel x86 Atom, android-29 | Intel x86 Atom_64, android-29 | Google APIs Intel x86 Atom, android-29 | Google APIs Intel x86 Atom_64, android-29 | Google Play Intel x86 Atom, android-29 | Google Play Intel x86 Atom_64, android-30 | Google APIs Intel x86 Atom, android-30 | Google APIs Intel x86 Atom_64, android-30 | Google Play Intel x86 Atom, android-30 | Google Play Intel x86 Atom_64
Android NDK: Not Found
Windows SDK:
AllowAllTrustedApps: Disabled
Versions: 10.0.17763.0
IDEs:
Android Studio: Version  <IP_ADDRESS> AI-193.69<IP_ADDRESS>26763
Visual Studio: Not Found
Languages:
Java: 1.8.0_262
Python: 3.8.6
npmPackages:
@react-native-community/cli: Not Found
react: 16.13.1 => 16.13.1
react-native: 0.63.3 => 0.63.3
react-native-windows: Not Found
npmGlobalPackages:
react-native: Not Found
Steps To Reproduce

npm install
npx react-native start
npx react-native run-android

Expected Results
The app should normaly compile and start.
Snack, code example, screenshot, or link to a repository:
[Mon Sep 28 2020 09:21:11.628]  BUNDLE  ./index.js
error: Error: Unable to resolve module ../components/input/Input from pages/ResetPasswordChange.js:
None of these files exist:

components/input/Input(.native|.android.js|.native.js|.js|.android.json|.native.json|.json|.android.ts|.native.ts|.ts|.android.tsx|.native.tsx|.tsx)
components/input/Input/index(.native|.android.js|.native.js|.js|.android.json|.native.json|.json|.android.ts|.native.ts|.ts|.android.tsx|.native.tsx|.tsx)
at ModuleResolver.resolveDependency (/home/aljosa/hometown/ipark/node_modules/metro/src/node-haste/DependencyGraph/ModuleResolution.js:163:15)
at ResolutionRequest.resolveDependency (/home/aljosa/hometown/ipark/node_modules/metro/src/node-haste/DependencyGraph/ResolutionRequest.js:52:18)
at DependencyGraph.resolveDependency (/home/aljosa/hometown/ipark/node_modules/metro/src/node-haste/DependencyGraph.js:287:16)
at Object.resolve (/home/aljosa/hometown/ipark/node_modules/metro/src/lib/transformHelpers.js:267:42)
at /home/aljosa/hometown/ipark/node_modules/metro/src/DeltaBundler/traverseDependencies.js:434:31
at Array.map ()
at resolveDependencies (/home/aljosa/hometown/ipark/node_modules/metro/src/DeltaBundler/traverseDependencies.js:431:18)
at /home/aljosa/hometown/ipark/node_modules/metro/src/DeltaBundler/traverseDependencies.js:275:33
at Generator.next ()
at asyncGeneratorStep (/home/aljosa/hometown/ipark/node_modules/metro/src/DeltaBundler/traverseDependencies.js:87:24)

@aljosazidan please try theses command:
rm -rf node_modules
npm install
npx jetify
cd android && ./gradlew clean && cd ..

@ps0305 thank you. This solved the problem.

@ps0305 thank you. my problem solved too


[EditableText] honor the "brieflyShowPassword" system setting

Fixes #65352

Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.
List which issues are fixed by this PR. You must list at least one issue.
If you had to change anything in the flutter/tests repo, include a link to the migration guide as per the breaking change policy.
Pre-launch Checklist

[ ] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[ ] I read the Tree Hygiene wiki page, which explains my responsibilities.
[ ] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[ ] I signed the CLA.
[ ] I listed at least one issue that this PR fixes in the description above.
[ ] I updated/added relevant documentation (doc comments with ///).
[ ] I added new tests to check the change I am making, or this PR is test-exempt.
[ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

This was reverted in https://github.com/flutter/flutter/pull/98089


Bug: Pop using AppBar X button returns to route without previous query params

Steps

Navigate to first route using some query parameter (e.g. {'X': 'true'})
Navigate to a stacked route (second) which is defined as a full-screen-dialog
Return to previous route using AppBar's X (close) button

Expected
Return to first route with the same query parameter that was previously set
Actual results
Return to first route with no query parameter set
Logs
[VRouter: INFO] Successfully navigated to "/" using VRouter.to
[VRouter: INFO] Successfully navigated to "/first?X=true" using VRouter.to
[VRouter: INFO] Successfully navigated to "/first/second" using VRouter.to
[VRouter: INFO] Successfully navigated to "/first" using VRouter.pop

Example project: https://github.com/amitkot/vrouter_bug

Thanks, historyBack works fine for my needs!
Why is the current behavior the intended one? Would you not expect a Navigator.pop() to return to the exact previous screen, with the same params?

I understand that your use case is different. However since the API has to:

Take everything into account
Be consistent (which is very important for easy use)

the current behavior is the best one.
If you need another argument, VRouter is not a state management library. It's important that the 2 are separate since VRouter would make a poor state management library, and having bits of it would not help. This is a strong choice but explains a lot of things (like why you can't pass an arbitrary state object during push), including this.
I hope you understand. If you need my advice on how to implement it I think:

historyBack is a good option if you are sure to only come from the route bellow
Otherwise I would use to('/pathBellow', queryParameters: Uri.parse(context.vRouter.previousUrl ?? '').queryParameters)

I understand your points, thanks!

historyBack is a good option when needing to navigate back, but is there an eloquent solution for when the user taps the phone's back button or presses the back button in the AppBar?

@baconcheese113 you can do:
return VRouter(
  //other params
   onSystemPop: (vRedirector) async {
              if (vRedirector.historyCanBack()) {
                vRedirector.historyBack();
              }
            },


Boulders at baba

Having the boulders disappear upon xp-drop would allow for more consistent boulder skipping when not everyone in ToA is as experienced with it.
Seeing the functionality is pretty much the same as with nylo's I could probably fork this plugin and submit it to the plugin hub, but maybe you want to incorporate them into this plugin (and maybe make the plugin name more generic in the process), in which case that'd also be nice just because it'd be one fewer plugin to install.
Mostly just making this issue to see what you would want me to do

Boulders disappearing upon xp drop would not be great because you can shoot them with a projectile that has a long delay and still get rolled tf out.
Another person is working on adding baba monkey waves and we'll rename this plugin, but I don't plan on accepting any other features.

Oh that's a good point actually


can't install Docker in remote host

After Installing Docker, the driver can't read the docker cmd. I assume that Docker is not been installed.
log is below
Creating machine...
(dev2) dev2 | Creating key pair for instance ...
(dev2) dev2 | Configuring security groups instance ...
(dev2) dev2 | Creating instance with image ubuntu1404_64_20G_aliaegis_20150325.vhd ...
(dev2) dev2 | Create instance i-23kov3n1j successfully
(dev2) dev2 | Allocate publice IP address <IP_ADDRESS> for instance i-23kov3n1j successfully
(dev2) dev2 | Starting instance i-23kov3n1j ...
(dev2) dev2 | Start instance i-23kov3n1j successfully
(dev2) dev2 | Waiting SSH service <IP_ADDRESS>:22 is ready to connect ...
(dev2) dev2 | Uploading SSH keypair to <IP_ADDRESS>:22 ...
(dev2) dev2 | Created instance i-23kov3n1j successfully with public IP address <IP_ADDRESS> and private IP address <IP_ADDRESS>
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with ubuntu(upstart)...
Installing Docker...
Error getting SSH command to check if the daemon is up: Something went wrong running an SSH command!
command : sudo docker version
err     : exit status 1
output  : sudo: docker: command not found

你可以设置环境变量来利用阿里云Docker Engine mirror来防止Docker安装失败
export MACHINE_DOCKER_INSTALL_URL= http://acs-public-mirror.oss-cn-hangzhou.aliyuncs.com/docker-engine/intranet
或者使用  --engine-install-url  参数


Split action loose selection state

Testing #213550

select some tabs
run any of the split actions

Observe: the split happens correctly, but after the split the editor from which I opened the context menu is selected. I would have expected that the multi select state is preserved.

One design decision we've made is when you have multiple tabs selected in one editor group and the move focus into another editor group, the selection of the previous group is cleared. This is what is happening in this case. This behaviour i not final yet. We are not dure yet what users would prefer.
@dbaeumer Would you expect that the selection is kept when switching focus between editor groups?

@benibenj yes that is what I would expect. It is very easy to clear a selection by single clicking on a tab.


how to add it to c++ program

i just want to know how is there a way to add it to c++ program so that it can detect language automatically.
Thanks

@tarinder6504 you can start by taking a look at the exposed public headers in the public folder to figure out which API you can use.
If you don't know where to start a good one would be to grep for ExtDetectLanguageSummary
Also, you will need the shared library:
To compile just cd in the internal folder and execute the compile_libs.sh script. That will give you a libcld2.so that can be linked.

thanks @fntlnz  i followed ur guidelines..when i execute compile_libs.sh I am getting error

@tarinder6504 which error?

@fntlnz
ld: unknown option: -soname=libcld2.so
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ld: unknown option: -soname=libcld2_full.so
clang: error: linker command failed with exit code 1 (use -v to see invocation)


memory fix

Build triggered. sha1 is merged.

Refer to this link for build results (access rights to CI server needed):
/job/libnd4j_PR/143/

Build finished. No test results found.


feat: create python script to import active events from meetup

Is your feature request related to a problem? Please describe.
Create python script to import data from meetup and include them into _data/events.yml
Describe alternatives you've considered
This script can be executed manually whenever necessary. Later the new events page will created and also could be categorised to be able to show in programmes, home (similar to announcements) and in respective new event Page.
Meetup Example event
Event example

Yml information

Title
Description
Date / hour
Hosters
Speakers
expiration (until when should be shown in home page as next event)
Category (Mentorship, Interview Prep, BookClub, Write Club and others)
Banner image link

Please check:

_data/events.yml
events html from home page

@lilyndk Could you create a new issue with the layout for events page.
Extra notes

This issue only covers the creation of yml file, not the events page.
Please consider create test for the script.

Screenshot as example

@lilyndk anything else you think will be necessary to be present in the events page?

@dricazenck @lilyndk - An idea, it should filter only the future events and not the past events.

@dricazenck What is meant by 'schedule' in the list under 'Yml information'?

@dricazenck @lilyndk - An idea, it should filter only the future events and not the past events.

Yep, I was thinking in have 2 sessions like next events and past events. But I like the idea of search for events. Similar to mentors search. So we can filter per category and etc. For this version I think would be good only have all events and we can create this as part of the platform project. What do you think @lilyndk @goelsonali ?

I think we can import all events past and upcoming and just separate them under 2 sections as you mentioned @dricazenck

We cannot import past events from old community. But we can have in the same events pages past and future events, yes.

@dricazenck @goelsonali Yes, agreed. We can add a tabs nav to separate the events list into upcoming and past events. And yes, I think we should keep it simple for the site with just listing the events. And for platform we can add search and filtering - see the Figma platform events page design here.

Theming task created: https://github.com/WomenCodingCommunity/WomenCodingCommunity.github.io/issues/232
PR created: https://github.com/WomenCodingCommunity/WomenCodingCommunity.github.io/pull/233


Improvement to PR #957 by creating DDAtomicCounter class

New Pull Request Checklist

[x] I have updated this branch with the latest master to avoid conflicts (via merge from master or rebase)
[x] I have added the required tests to prove the fix/feature I am adding
[x] I have updated the documentation (if necessary)
[x] I have run the tests and they pass
[ ] I have run the lint and it passes (pod lib lint)

This merge request fixes / refers to the following issues: #957
Pull Request Description
Improvement to PR #957 Replace OSAtomic with stdatomic in DDDispatchQueueLogFormatter
Created a DDAtomicCounter class with DDAtomicCountable protocol that encapsulates the atomic counting logic.
Using another macro DD_OSATOMIC_API_DEPRECATED instead of all the checks for min os version
Cleaned up the imports of OSAtomic.h
Added some basic tests of this new class

CC @CocoaLumberjack/collaborators

Codecov Report

Merging #958 into master will increase coverage by 1.76%.
The diff coverage is 92.42%.

@@            Coverage Diff             @@
##           master     #958      +/-   ##
==========================================
+ Coverage   22.22%   23.99%   +1.76%
==========================================
  Files          17       18       +1
  Lines        2641     2701      +60
==========================================
+ Hits          587      648      +61
+ Misses       2054     2053       -1

Impacted Files
Coverage Δ

Classes/DDLog.m
55.01% <ø> (ø)
:arrow_up:

Classes/Extensions/DDDispatchQueueLogFormatter.h
0% <ø> (ø)
:arrow_up:

Classes/DDFileLogger.m
0% <ø> (ø)
:arrow_up:

Classes/Extensions/DDDispatchQueueLogFormatter.m
7.93% <71.42%> (+7.93%)
:arrow_up:

Tests/Tests/DDAtomicCounterTests.m
98.07% <98.07%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0ddce26...70738e4. Read the comment docs.

Thanks @sushichop


Add scraper for Virginia County Level Hospitalizations & Death Data

Location name
All County/City, Virginia, USA
Source URL
http://www.vdh.virginia.gov/content/uploads/sites/182/2020/03/VDH-COVID-19-PublicUseDataset-Cases.csv
Notes/comments
Released each morning by 10:00 am eastern

From Virginia Department of Health:
UPDATE: 4/23/20 - In addition to today's counts, this data set now includes historical data of cumulative counts by date
VDH COVID-19-PublicUseDataset-Cases


consider base integration branch config on i:Take

Q
A

Fixed Tickets

License
MIT

A new inspection was created.

The inspection completed: No new issues

A new inspection was created.

The inspection completed: No new issues

A new inspection was created.

The inspection completed: No new issues


Try to do some basic machine learning on my uLogMe data, see what I can get?

Try to do some basic machine learning on my uLogMe data, see what I can get?

Hello, this so great to have your good working uLogMe fork! Don't have that much time to dig in, but here are some quickly thought ideas:

Find your most productive pattern (hours, days, ideal work sequence, do you need breaks?, what's your best type of break? Away from keyboard or scrolling Social Networks?
 Email? Kernel panic?)
Predictive alert: trigger an alert if your about to procrastinate (Let's detect bad patterns: Professional mail answering -> perso answering -> social networks).
Go deeper: add new inputs. Weather sensor. Location sensor. A live webcam input (front facing laptop cam) to infer in what posture, facial expression, hairstyle, you are the most productive...

Which ask a big question:

What is a good productivity metric?
On this, I like a bayesian productivity approach where you can assign percent scores to each app you use (ex: 0% for social networks, 100% for Terminal). This is what I'm used to with the macOs Timing app. Also App usage can be aggregate to Task, then you can assign score to Task.

Finally, we could all anonymize and upload our collected data, then concat to create the biggest worldwide dataset ever.. Let's answering crucial questions like, are the human more productive when it rains? Are french the most productive? So much coding to go.

Hi,
Who there is a lot of ideas here… thanks!
Thanks for your interest in uLogMe!
I will definitely never have time to code anything as serious.
Plus, uLogMe is not so popular. Other time trackers have way more users, see for instance https://wakatime.com/

I'll try to reply to your questions and ideas as soon as I have some free time…

Hi @a1rb4Ck, sorry, of course I never had enough time to come back at this, and never thought about it again.
I'm just dropping here some links about a very impressive framework built by someone:
https://beepb00p.xyz/hpi.html, https://beepb00p.xyz/my-data.html, https://beepb00p.xyz/pkm-search.html
And by someone else: https://github.com/KrauseFx/FxLifeSheet


[BACKLOG-21882] - Enable pdi log to roll to next day.

Map PDI log level to log4j threshold.
Update format of the log entry.

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'core' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  assemblies/lib/pom.xml
  assemblies/static/src/main/resources/classes/log4j.xml
  core/src/main/java/org/pentaho/di/core/logging/Slf4jLoggingEventListener.java

Newly Broken Tests:
org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventJob:
:red_circle: org.mockito.exceptions.verification.junit.ArgumentsAreDifferent
org.mockito.exceptions.verification.junit.ArgumentsAreDifferent:

Argument(s) are different! Wanted:
jobLogger.info("filename message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventJob(Slf4jLoggingEventListenerTest.java:111)
Actual invocation has different arguments:
jobLogger.info("[filename]  message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListener.logToLogger(Slf4jLoggingEventListener.java:96)

	at org.pentaho.di.core.logging.Slf4jLoggingEvent

org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventTrans:
:red_circle: org.mockito.exceptions.verification.junit.ArgumentsAreDifferent
org.mockito.exceptions.verification.junit.ArgumentsAreDifferent:

Argument(s) are different! Wanted:
transLogger.info("filename message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventTrans(Slf4jLoggingEventListenerTest.java:93)
Actual invocation has different arguments:
transLogger.info("[filename]  message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListener.logToLogger(Slf4jLoggingEventListener.java:96)

	at org.pentaho.di.core.logging.Slf4jLogging

Still Broken Tests:
org.pentaho.di.trans.steps.syslog.SyslogMessageConcurrentTest.concurrentSyslogMessageTest:
:red_circle: java.lang.Exception
java.lang.Exception: test timed out after 10000 milliseconds
	at org.pentaho.di.trans.steps.syslog.SyslogMessageConcurrentTest.createSyslogMessageTask(SyslogMessageConcurrentTest.java:118)
	at org.pentaho.di.trans.steps.syslog.SyslogMessageConcurrentTest.concurrentSyslogMessageTest(SyslogMessageConcurrentTest.java:67)

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.core.logging.Slf4jLoggingEventListener

Branch Change: -8.4% :small_red_triangle_down:
Complexity Change: -8.6% :small_red_triangle_down:
Instruction Change: -2.5% :small_red_triangle_down:
Line Change: -4.4% :small_red_triangle_down:
Method Change: -3.6% :small_red_triangle_down:

org.pentaho.di.core.injection.bean.BeanInjector

Branch Change: -0.7% :small_red_triangle_down:
Complexity Change: -1.2% :small_red_triangle_down:

org.pentaho.di.job.entries.evaluatetablecontent.JobEntryEvalTableContent

Branch Change: -1% :small_red_triangle_down:
Complexity Change: -1% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.job.entries.sql.JobEntrySQL

Instruction Change: -3.3% :small_red_triangle_down:
Line Change: -3.9% :small_red_triangle_down:

org.pentaho.di.trans.step.BaseStep

Branch Change: -0.7% :small_red_triangle_down:
Complexity Change: -0.6% :small_red_triangle_down:
Instruction Change: -0.5% :small_red_triangle_down:
Line Change: -0.7% :small_red_triangle_down:
Method Change: -0.9% :small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Branch Change: -0.7% :small_red_triangle_down:
Complexity Change: -0.6% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:
Line Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.step.StepStatus

Branch Change: -3.3% :small_red_triangle_down:
Complexity Change: -1.6% :small_red_triangle_down:
Instruction Change: -1.4% :small_red_triangle_down:

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change: -1.1% :small_red_triangle_down:
Complexity Change: -1.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta

Branch Change: -2.6% :small_red_triangle_down:
Complexity Change: -1.5% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta

Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.salesforceinsert.SalesforceInsertMeta

Instruction Change: -0.1% :small_red_triangle_down:

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'core' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  assemblies/lib/pom.xml
  assemblies/static/src/main/resources/classes/log4j.xml
  core/src/main/java/org/pentaho/di/core/logging/Slf4jLoggingEventListener.java
  core/src/test/java/org/pentaho/di/core/logging/Slf4jLoggingEventListenerTest.java

Newly Broken Tests:
org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventJob:
:red_circle: org.mockito.exceptions.verification.WantedButNotInvoked
org.mockito.exceptions.verification.WantedButNotInvoked:

Wanted but not invoked:
jobLogger.error("[filename]  message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventJob(Slf4jLoggingEventListenerTest.java:115)

However, there were other interactions with this mock:
jobLogger.info("[filename]  message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListener.logToLogger(Slf4jLoggingEventListener.java:96)

jobLogger.info("[filename]  message");
-> at org.pe

org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventTrans:
:red_circle: org.mockito.exceptions.verification.WantedButNotInvoked
org.mockito.exceptions.verification.WantedButNotInvoked:

Wanted but not invoked:
transLogger.error("[filename]  message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventTrans(Slf4jLoggingEventListenerTest.java:97)

However, there were other interactions with this mock:
transLogger.info("[filename]  message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListener.logToLogger(Slf4jLoggingEventListener.java:96)

transLogger.info("[filename]  message");
-> at

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.core.logging.Slf4jLoggingEventListener

Branch Change: -8.4% :small_red_triangle_down:
Complexity Change: -8.6% :small_red_triangle_down:
Instruction Change: -2.5% :small_red_triangle_down:
Line Change: -4.4% :small_red_triangle_down:
Method Change: -3.6% :small_red_triangle_down:

org.pentaho.di.job.entries.sql.JobEntrySQL

Instruction Change: -3.3% :small_red_triangle_down:
Line Change: -3.9% :small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Branch Change: -0.7% :small_red_triangle_down:
Complexity Change: -0.6% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:
Line Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Branch Change: -2% :small_red_triangle_down:
Complexity Change: -1.7% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.dbproc.DBProcMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.ifnull.IfNullMeta

Branch Change: -2% :small_red_triangle_down:
Complexity Change: -1.9% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change: -2.2% :small_red_triangle_down:
Complexity Change: -2.5% :small_red_triangle_down:
Instruction Change: -0.3% :small_red_triangle_down:
Line Change: -0.9% :small_red_triangle_down:

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta

Branch Change: -0.9% :small_red_triangle_down:
Complexity Change: -0.8% :small_red_triangle_down:

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta

Branch Change: -0.5% :small_red_triangle_down:

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.salesforceupsert.SalesforceUpsertMeta

Instruction Change: -0.1% :small_red_triangle_down:

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'core' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  assemblies/lib/pom.xml
  assemblies/static/src/main/resources/classes/log4j.xml
  core/src/main/java/org/pentaho/di/core/logging/Slf4jLoggingEventListener.java
  core/src/test/java/org/pentaho/di/core/logging/Slf4jLoggingEventListenerTest.java

Newly Broken Tests:
org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventJob:
:red_circle: org.mockito.exceptions.verification.WantedButNotInvoked
org.mockito.exceptions.verification.WantedButNotInvoked:

Wanted but not invoked:
jobLogger.error("[filename]  message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventJob(Slf4jLoggingEventListenerTest.java:115)

However, there were other interactions with this mock:
jobLogger.info("[filename]  message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListener.logToLogger(Slf4jLoggingEventListener.java:96)

jobLogger.info("[filename]  message");
-> at org.pe

org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventTrans:
:red_circle: org.mockito.exceptions.verification.WantedButNotInvoked
org.mockito.exceptions.verification.WantedButNotInvoked:

Wanted but not invoked:
transLogger.error("[filename]  message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListenerTest.testAddLogEventTrans(Slf4jLoggingEventListenerTest.java:97)

However, there were other interactions with this mock:
transLogger.info("[filename]  message");
-> at org.pentaho.di.core.logging.Slf4jLoggingEventListener.logToLogger(Slf4jLoggingEventListener.java:96)

transLogger.info("[filename]  message");
-> at

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.core.logging.Slf4jLoggingEventListener

Branch Change: -8.4% :small_red_triangle_down:
Complexity Change: -8.6% :small_red_triangle_down:
Instruction Change: -2.5% :small_red_triangle_down:
Line Change: -4.4% :small_red_triangle_down:
Method Change: -3.6% :small_red_triangle_down:

org.pentaho.di.job.entries.evalfilesmetrics.JobEntryEvalFilesMetrics

Branch Change: -0.4% :small_red_triangle_down:
Complexity Change: -0.5% :small_red_triangle_down:

org.pentaho.di.job.entries.getpop.MailConnectionMeta

Branch Change: -1% :small_red_triangle_down:
Complexity Change: -1.5% :small_red_triangle_down:
Instruction Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.checksum.CheckSumMeta

Branch Change: -1.2% :small_red_triangle_down:
Complexity Change: -1.4% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.csvinput.CsvInputMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta

Method Change: -1% :small_red_triangle_down:

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change: -1.1% :small_red_triangle_down:
Complexity Change: -1.3% :small_red_triangle_down:

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'core' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  assemblies/lib/pom.xml
  assemblies/static/src/main/resources/classes/log4j.xml
  core/src/main/java/org/pentaho/di/core/logging/Slf4jLoggingEventListener.java
  core/src/test/java/org/pentaho/di/core/logging/Slf4jLoggingEventListenerTest.java

Newly Broken Tests:
org.pentaho.di.trans.streaming.common.BlockingQueueStreamSourceTest.testRowsFilled:
:red_circle: junit.framework.AssertionFailedError
junit.framework.AssertionFailedError
	at org.pentaho.di.trans.streaming.common.BlockingQueueStreamSourceTest.getQuickly(BlockingQueueStreamSourceTest.java:225)
	at org.pentaho.di.trans.streaming.common.BlockingQueueStreamSourceTest.testRowsFilled(BlockingQueueStreamSourceTest.java:169)

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.core.logging.Slf4jLoggingEventListener

Branch Change: -8.4% :small_red_triangle_down:
Complexity Change: -8.6% :small_red_triangle_down:
Instruction Change: -2.5% :small_red_triangle_down:
Line Change: -4.4% :small_red_triangle_down:
Method Change: -3.6% :small_red_triangle_down:

org.pentaho.di.job.entries.evaluatetablecontent.JobEntryEvalTableContent

Branch Change: -1% :small_red_triangle_down:
Complexity Change: -1% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.job.entries.getpop.MailConnectionMeta

Branch Change: -1% :small_red_triangle_down:
Complexity Change: -1.5% :small_red_triangle_down:
Instruction Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.step.BaseStep

Branch Change: -0.1% :small_red_triangle_down:
Complexity Change: -0.2% :small_red_triangle_down:
Line Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Branch Change: -0.7% :small_red_triangle_down:
Complexity Change: -0.6% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:
Line Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.calculator.CalculatorMetaFunction

Branch Change: -8.7% :small_red_triangle_down:
Complexity Change: -3% :small_red_triangle_down:
Instruction Change: -0.7% :small_red_triangle_down:
Line Change: -0.9% :small_red_triangle_down:

org.pentaho.di.trans.steps.csvinput.CsvInputMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Branch Change: -2% :small_red_triangle_down:
Complexity Change: -1.7% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.dbproc.DBProcMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.processfiles.ProcessFilesMeta

Branch Change: -3.5% :small_red_triangle_down:
Complexity Change: -2.1% :small_red_triangle_down:
Instruction Change: -0.4% :small_red_triangle_down:

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.rowgenerator.RowGenerator

Branch Change: -7.1% :small_red_triangle_down:
Complexity Change: -5.6% :small_red_triangle_down:
Instruction Change: -2.8% :small_red_triangle_down:
Line Change: -3.7% :small_red_triangle_down:

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta

Branch Change: -2.6% :small_red_triangle_down:
Complexity Change: -1.5% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.setvalueconstant.SetValueConstantMeta

Instruction Change: -0.2% :small_red_triangle_down:

Note: The list of offending coverage changes extends beyond the presented results and was truncated to prevent reaching the comment size limit.

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'core' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'core' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  assemblies/lib/pom.xml
  assemblies/static/src/main/resources/classes/log4j.xml
  core/src/main/java/org/pentaho/di/core/logging/Slf4jLoggingEventListener.java
  core/src/test/java/org/pentaho/di/core/logging/Slf4jLoggingEventListenerTest.java

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.core.logging.Slf4jLoggingEventListener

Branch Change: -8.4% :small_red_triangle_down:
Complexity Change: -8.6% :small_red_triangle_down:
Instruction Change: -2.5% :small_red_triangle_down:
Line Change: -4.4% :small_red_triangle_down:
Method Change: -3.6% :small_red_triangle_down:

org.pentaho.di.job.entries.evaluatetablecontent.JobEntryEvalTableContent

Branch Change: -1% :small_red_triangle_down:
Complexity Change: -1% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.job.entries.getpop.MailConnectionMeta

Branch Change: -1% :small_red_triangle_down:
Complexity Change: -1.5% :small_red_triangle_down:
Instruction Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change: -1% :small_red_triangle_down:
Complexity Change: -1.1% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:
Line Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.insertupdate.InsertUpdateMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.www.cache.CarteStatusCache

Instruction Change: -1.2% :small_red_triangle_down:
Line Change: -1.3% :small_red_triangle_down:

@pentaho/moseisley  please review


DeviceController: refactor pairing

Summary of Changes
Remove RendezvousSession usage in device contorller
RendezvousSession handles SPAKE2 pairing session and network provision, but network provision will be removed from it, so it is only handling SPAKE2 pairing session. In this case, it is better to use PASESession directly.
Move DeviceCommissioner functionalities to Device
DeviceCommissioner is a singleton class handling pairing session, it is basically bind to a device while doing a the pairing procedure, and the execution flow is bouncing between the device and the commissioner. When moving all these code in to the Device class, the code become more clear and understandable.
I think the original code is highly inspired by the old device controller, which handles only one device. But the new device controller handles multiple devices, and the device pairing part belongs to a device instead a global controller.
Note
This PR will break network provision and BLE pairing.
Network provision: current implementation doesn't align with the spec, it need to be a cluster instead.
BLE pairing: @erjiaqing will bring a unified transport for both BLE and IP, it will be fix then.

@pan-apple ?

I am a bit confused with the design change this PR is bringing.

What's the use of an unpaired CHIPDevice object?
Once a device has been paired, the pairing session and corresponding information used for pairing process is not really needed. The updated CHIPDevice object will carry it now.
Overall, I am not sure what problem we are solving by moving DeviceCommissioner functionality into CHIPDevice object. Maybe there're better ways to solve it.

I am a bit confused with the design change this PR is bringing.

What's the use of an unpaired CHIPDevice object?

A device waiting to be pairing, it is just an abstraction.

Once a device has been paired, the pairing session and corresponding information used for pairing process is not really needed. The updated CHIPDevice object will carry it now.

Yes, and the Channel API #4019 will solved the problem. The pairing information will be saved in the channel builder, the throw out once pairing is completed.

Overall, I am not sure what problem we are solving by moving DeviceCommissioner functionality into CHIPDevice object. Maybe there're better ways to solve it.

Yes, the Channel API #4019 is the better way.
Overall, everything suggesting that #4019 is blocking lots of thing, please priorities #4019 first.

Note
This PR will break network provision and BLE pairing.
Network provision: current implementation doesn't align with the spec, it need to be a cluster instead.
BLE pairing: @erjiaqing will bring a unified transport for both BLE and IP, it will be fix then.

@kghost Thank you for your work! I didn't review the content in the details but would like to express my opinion on the above fragment of PR description.
In my opinion, this is not an acceptable approach, to introduce such a big regression (lost of features) to get a value of better code. As a Thread developer, I wouldn't be able to perform Rendezvous and Thread provisioning at all (if I read the description correctly). All examples will stop working for many people (including CHIP Harness team) and so developers/early adopters would need to do some hacks to test ZCL communication with the Phone. While I agree, the current version of Network Provisioning is not the final one, we should iteratively replace it with the new one without breaking master for a couple of days/weeks.
Could we first introduce the cluster-based Network Provisioning, and (only then/at the same time) start refactoring the code?

Is it possible that we first introduce the cluster-based Network Provisioning, and (only then/at the same time) start refactoring the code?

@erjiaqing is working on that part, it will probably land soon.

Yes, had a chat on slack with him. Thank you @kghost!

Blocked by #3945 #4421

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Determining what to do with startActionListener()

We need to figure out how the functionality of startActionListener() is going to work for the SDK.
@owaiskazi19 The normal answer to that question is that it doesn't affect any instance variables so it would help the compiler (potentially) optimize.  However...
Why doesn't it affect any instance variables?  It looks like we instantiate an ActionListener on the next line, tell it to run, and then promptly forget about it, so we can't get any info from it again, and it is eligible for garbage collection unless one of its internal variables has a reference retained.
Which it doesn't.  It does open a ServerSocket which looks internally like it starts a thread... which dies on handshake or after an hour, but that's not attached to any object so it looks like it just floats out there for a while, keeping the socket alive.
There doesn't appear to be any way to prevent multiple action listeners from simultaneously executing, which in the case of a static method is possibly a good thing, as each of multiple extensions may start one up... except that after starting them up they have no reference to it anymore.
All this is to say this should probably not be static, but there's a bigger problem of not keeping track of the action listener at all. Shouldn't it be an instance variable and shouldn't this method only start it if it hasn't already been started?  Or as part of a constructor?
I realize this problem existed before this PR, but I suspect the failure will never be triggered (or if it is, nobody will get the result of it).
Originally posted by @dbwiddis in https://github.com/opensearch-project/opensearch-sdk-java/pull/87#discussion_r966394193

@ryanbogan the comment mentioned by @dbwiddis was related to the static keyword and not startActionListener. I have mentioned here about the other APIs as well. To frame in a better way:
Once we integrate SDK with AD Extension, we will call createIndex from SDKClient and if the request fails an exception will thrown on AD Extension end.
To have the similar exception thrown as well on OpenSearch logs we will call the startActionListener() API while creating Index in the catch block. I hope this clear the issue and we can close it out.

This issue is primarily a "don't forget we need to eventually do something with this listener" signpost. Not sure anything can be done yet but let's not forget about it.

We should change this issue to address all the methods then rather just startActionListener.
sendClusterStateRequest, sendClusterSettingsRequest, sendLocalNodeRequest and sendRegisterRestActionsRequest. @ryanbogan will you edit the issue then?

I think the correct solution to this problem is to sequence all these requests inside the ExtensionsInitRequestHandler.   See for example how sendRegisterRestActionsRequest() (already done) and sendEnvironmentSettingsRequest() (part of #244) fits nicely into an initialization sequence:
try {
    return new InitializeExtensionsResponse(extensionsRunner.settings.get(NODE_NAME_SETTING));
} finally {
    // After sending successful response to initialization, send the REST API and Settings
    extensionsRunner.setOpensearchNode(extensionsRunner.opensearchNode);
    extensionsRunner.setExtensionNode(extensionInitRequest.getExtension());
    TransportService extensionTransportService = extensionsRunner.getExtensionTransportService();
    extensionTransportService.connectToNode(extensionsRunner.opensearchNode);
    extensionsRunner.sendRegisterRestActionsRequest(extensionTransportService);
    extensionsRunner.sendRegisterCustomSettingsRequest(extensionTransportService);
    extensionsRunner.transportActions.sendRegisterTransportActionsRequest(
        extensionTransportService,
        extensionsRunner.opensearchNode,
        extensionsRunner.getUniqueId()
    );
    // Get OpenSearch Settings and set values on ExtensionsRunner
    Settings settings = extensionsRunner.sendEnvironmentSettingsRequest(extensionTransportService);
    extensionsRunner.setEnvironmentSettings(settings);
    extensionsRunner.setNamedXContentRegistry(
        new ExtensionNamedXContentRegistry(settings, extensionsRunner.getCustomNamedXContent())
    );

    // Last step of initialization
    extensionsRunner.setInitialized();
}

Adding the setInitialized() at the end as I did in #244 also resolves my concern in #94.
Recommend this issue and #94 be closed when #244 is merged.

@dbwiddis sendClusterStateRequest, sendClusterSettingsRequest andsendLocalNodeRequest are APIs which should be triggered by any extension only when it's needed. We should not add these methods in InitializeExtensionsResponse.


[boost-stacktrace] build failure when building dependencies from vcpkg manifest in arm64-windows triplet

Host Environment

OS: Windows
Compiler: revision

To Reproduce
Steps to reproduce the behavior:

Create a C++ console application in visual studio and enable "Use vcpkg manifest" for the project.
Specify boost-stacktrace as one of the dependencies in vcpkg manifest file.
Switch build configuration to "ARM64" platform and try building the project in visual studio.

Failure logs
CMake Error at installed/arm64-windows/share/boost-build/boost-modular-build.cmake:47 (message):
  The x86 boost-build tools must be installed to build for non-x86/x64
  platforms.  Please run `vcpkg install boost-build:x86-windows`.
Call Stack (most recent call first):
  ports/boost-stacktrace/portfile.cmake:12 (boost_modular_build)
  scripts/ports.cmake:136 (include)

Additional context
As the error message specifies boost-build:x86-windows is required to build boost-stacktrace:arm64-windows. Disabling "Use vcpkg manifest" and then after that manually installing boost-build:x86-windows and then boost-stacktrace:arm64-windows works fine. But enabling "Use vcpkg manifest" always results in build failure for ARM64.

Got the same error on linux building arm architecture , one quick fix is modifying the installed/arm64-windows/share/boost-build/boost-modular-build.cmake file to set the correct boost path

@soumyamahunt, thanks for reporting this issue, please try the workround that @DiegoAvila123 provided.
It's a similar issue with https://github.com/microsoft/vcpkg/issues/15087. I will use that issue to track.

@PhoebeHui I tried building with latest vcpkg commit and build for arm64-windows triplet now succeeds but for arm-windows triplet it fails. May be whatever work done to pass the build for arm64-windows triplet could be done for arm-windows triplet too.


Microsoft.DigitalTwins - add stable data-plane API version 2022-05-31

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
If updating an existing version, please select the specific language SDKs and CLIs that must be refreshed after the swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[x] No refresh required for updates in this PR

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that label "WaitForARMFeedback" will be added automatically to begin ARM API Review. Failure to comply may result in delays to the manifest.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you are using OpenAPIHub to initialize the PR for adding a new version. More details, refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If any of the following scenarios apply to the PR, request approval from the Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in a stable version
[ ] Removing properties in a stable version
[ ] Removing API version(s) in a stable version
[ ] Updating API in a stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

To help drive the review, here is a summary of the changes from the previous version (2021-06-30-preview):

Updated API version header
Removed enumeration literals from api-version parameter, and converted to string per past guidance
Added "sourceTime" usage to examples

Note: I don't know what to do about the ModelValidation failure. We have used an "object" return type in the past, since the API response greatly depends on a model defined by the customer.

@jianyexi for ModelValidation failure. At least the error message does not make sense for INVALID_TYPE. Expected type {0} but found type {1}. but it does not say anything for expected type.

@jianyexi for ModelValidation failure. At least the error message does not make sense for INVALID_TYPE. Expected type {0} but found type {1}. but it does not say anything for expected type.

@raych1  to confirm if it's a bug

"required" in schema should be done like this: https://github.com/Azure/azure-rest-api-specs/blob/6d6b7c15deab7ae76ed51ca9c2f20c13cca57065/specification/digitaltwins/data-plane/Microsoft.DigitalTwins/stable/2022-05-31/digitaltwins.json#L1245-L1248

"required" in schema should be done like this:
https://github.com/Azure/azure-rest-api-specs/blob/6d6b7c15deab7ae76ed51ca9c2f20c13cca57065/specification/digitaltwins/data-plane/Microsoft.DigitalTwins/stable/2022-05-31/digitaltwins.json#L1245-L1248

This has now been addressed.

One thing to note - I've removed the "readyOnly" attribute from the error response model parameters because one of the automated validation tools complained that a required property should not be readyonly. This doesn't quite make sense to me - doesn't readonly set only the context whether it is from a response or a request? Should / can this be fixed?

@sjiherzig
My fault. OpenAPI 2.0 does not allow specifying both "readonly" and "required" to property. (see https://swagger.io/specification/v2/ on "readonly")
Our guide https://github.com/microsoft/api-guidelines/blob/vNext/azure/Guidelines.md#handling-errors and some SDK weight more on "required", as in this particular case, "readonly" of the whole "Error" schema can be deduced (as "Error" is not used as input in any operation).
In this case, maybe not adding the "required" is the base choice, as it keep it same as your last GA api-version. But do keep it on "ErrorResponse".

@sjiherzig
Please also note that the guideline asks for the "x-ms-error-code" header on error response.
https://github.com/microsoft/api-guidelines/blob/vNext/azure/Guidelines.md#handling-errors
Please do evaluate whether it can be included in this api-version (it requires backend change). If yes, please support it; if not, please open a backlog item in your service to add it in next api-version.

Hi @weidongxu-microsoft - this is now ready to be merged!

@sjiherzig
Do we have an issue relates to the PR?
I didn't remember since it is a long time ago. It is possible you do not have the issue and meeting notes, as there is no API change since last version.

@weidongxu-microsoft Yes, there is a GitHub issue for this PR: https://github.com/Azure/azure-rest-api-specs/issues/18792 .
There are no outstanding action items for the PR. The review was done asynchronously.


doc: Clarify MultiMesh.set_instance_color re: white albedo color

If the user not set the albedo color to pure white, then the multiplication with a diferent value than 1 will produce color totally wrong.
The pure white color r 255, g 255, b 255 internaly in Godot is 1 then the multiplication work ok.

Pushed an update with Calinou's edit and a reworded commit message for clarity.

Thanks! And congrats for your first merged Godot contribution :tada:

Cherry-picked for 3.6.

Cherry-picked for 3.5.1.


library not found for -lAFNetworking

I tried to install AFNetworking as a Pod with Cocoapods v0.38.2 but XCode always shows me this error
ld: library not found for -lAFNetworking clang: error: linker command failed with exit code 1 (use -v to see invocation)
This is not the first time that I try to use AFNetworking. Infact some time ago it worked perfectly but from some days it started to give me that error because i reinstalled the pods running command
pod install
Maybe the problem is not the AFNetworking but something on my project settings, but I didn't change anything.
Can someone help me?

What version of Xcode are you using? This is most likely a Cocoapods issue, not an AFNetworking issue.
Can you push up an example to Github demonstrating the problem? Does it work for fresh project?

I have the same problem with XCode 7.1 and XCode 6.4.
It seems like cocoapods issue as you said or maybe something setted wrong on my project.
What can determinate this error?

I'd go down the troubleshooting steps of cocoapods version checking, installing it in a clean project, reviewing project settings, etc. This issue falls outside the scope of the AFNetworking project, so I am going to close this one out. I'd recommend directing your question to forum more suited for support, like StackOverflow. We use the issues list on Github for tracking bugs and feature requests directly related to AFNetworking.
:beers:

Did you find a solution for this problem? Im also facing the same issue

Not yet. I tried to install AFNetworking pod with varius version of cocoapods but nothing solve my problem. I'm starting to think maybe it depends from versione of iOS target version of my application, but I can't change it because i need it to run from specific version of iOS.
Have you any ideas @gholias ?

No and it is really frustrating...
I think I will have to add AFNetworking without cocoapods

its because of use_framework.

manpreetg <---------- He is right!!!-----------> it`s because of use_framework.


[0.24.0-rc5][Android][NativeAnimatedModule]Native animated module is not available

I am getting the following error in RN 0.24.0-rc5 - Native animated module is not available
Animated.timing(     this.state.anim, { duration: 200, toValue: 1, useNativeDriver: true } ).start();
I am using it to update opacity only.
Note that without useNativeDrive: true the animation works fine.

useNativeDriver might not be supported -- just leave it off for now. Will be fixed in a release probably 2-8 weeks from now.

any news here? When does the NativeAnimatedModule arrive?

@ms88privat did you manage to enable this module?, i'm having issues enabling NativeAnimatedModule.

@nschurmann no, I didn't get it to work too. Keep me updated if you find a way, thanks!

@nschurmann did u figure it out how to include nativeanimatedmodule on android?

@sibelius It is now included by default. Just add useNativeDriver: true.


When opening extension URI, URL should be URI, and URI should be shown in full

Transferred from internal repo, when testing the vscode.env.createAppUri API

These are URI not URL. I'm also wondering why we are showing the ... when we have enough space to display the full URI.

The cropping is hard since we need to accommodate for all the different dialogs we have: web, mac, win, linux. We simply have to pick a number. We currently crop at 40 characters.


[Concurrency] Remove OPTNONE workaround for old clang version

This reverts c07aa9c425ff96365683e8b3915f23452bb03b6b which was done to avoid a crash in optimnized caused by this PR:
https://github.com/apple/swift/pull/41088
Since this was almost 2 years ago, we probably don't have this issue anymore as far as I can see other resolved issues, so try to remove the workaround.
Resolves rdar://88711954

@swift-ci please test


Could you provide some missing files and double-check the meta-model checkpoint?

The README mentions the argument --config finetuning-base in the fine-tuning step and a training configuration configs/default.yaml in the training step. I suppose the config directory was not committed.
The preprocessing script uses a file inference_folder.py for Graphonomy, is it a custom script modified from the original inference.py? If so, could you provide the script?
Without the finetuning-base config, I manually added the --finetune argument for fine-tuning but encountered the following errors. How should I resolve it?
PID 10490 - 2020-10-29 11:14:39,776 - INFO - utils.load_config_file - Using config configs/finetuning-base.yaml
PID 10490 - 2020-10-29 11:14:39,776 - WARNING - utils.get_args_and_modules - Could not load config finetuning-base
PID 10490 - 2020-10-29 11:14:39,776 - INFO - utils.get_args_and_modules - Loading checkpoint file checkpoints/latent-pose-release.pth
PID 10490 - 2020-10-29 11:14:41,345 - INFO - utils.setup - Random Seed: 123
PID 10490 - 2020-10-29 11:14:42,995 - INFO - train.py - Initialized the process group, my rank is 0
PID 10490 - 2020-10-29 11:14:42,995 - WARNING - train.py - Sorry, multi-GPU fine-tuning is NYI, setting `--num_gpus=1`
PID 10490 - 2020-10-29 11:14:42,995 - INFO - train.py - Loading dataloader 'voxceleb2_segmentation_nolandmarks'
PID 10490 - 2020-10-29 11:14:43,133 - INFO - dataloaders.common.voxceleb.get_part_data (train) - Determining the 'train' data source
PID 10490 - 2020-10-29 11:14:43,133 - INFO - dataloaders.common.voxceleb.get_part_data (train) - Checking if '/path/latent-pose-reenactment/data/VoxCeleb1_test_finetuning/images-cropped/id10280/XiKRlssBw2M/000330#001148.mp4' is a directory...
PID 10490 - 2020-10-29 11:14:43,133 - INFO - dataloaders.common.voxceleb.get_part_data (train) - Yes, it is; the only train identity will be 'id10280/XiKRlssBw2M/000330#001148.mp4'
PID 10490 - 2020-10-29 11:14:43,138 - INFO - dataloaders.common.voxceleb.get_part_data (train) - This dataset has 818 images
PID 10490 - 2020-10-29 11:14:43,139 - INFO - dataloaders.common.voxceleb.get_part_data (train) - Setting `args.num_labels` to 1 because we are fine-tuning or the model has been fine-tuned
PID 10490 - 2020-10-29 11:14:43,148 - WARNING - dataloader - Could not find the '.npy' file with bboxes, will assume the images are already cropped
PID 10490 - 2020-10-29 11:14:43,148 - INFO - dataloaders.augmentation - Pixelwise augmentation: True
PID 10490 - 2020-10-29 11:14:43,148 - INFO - dataloaders.augmentation - Affine scale augmentation: True
PID 10490 - 2020-10-29 11:14:43,148 - INFO - dataloaders.augmentation - Affine shift augmentation: True
PID 10490 - 2020-10-29 11:14:43,160 - INFO - dataloaders.dataloader - This process will receive a dataset with 409 samples
PID 10490 - 2020-10-29 11:14:43,160 - INFO - train.py - Starting from checkpoint checkpoints/latent-pose-release.pth
PID 10490 - 2020-10-29 11:14:43,160 - INFO - utils.load_model_from_checkpoint - Loading embedder 'unsupervised_pose_separate_embResNeXt_segmentation'
PID 10490 - 2020-10-29 11:14:44,027 - INFO - utils.load_model_from_checkpoint - Loading generator 'vector_pose_unsupervised_segmentation_noBottleneck'
PID 10490 - 2020-10-29 11:14:44,552 - INFO - utils.load_model_from_checkpoint - Loading discriminator 'no_landmarks'
PID 10490 - 2020-10-29 11:14:45,501 - WARNING - utils.load_model_from_checkpoint - Discriminator has changed in config (maybe due to finetuning), so not loading `optimizer_D`
PID 10490 - 2020-10-29 11:14:45,501 - INFO - utils.load_model_from_checkpoint - Loading runner holycow
PID 10490 - 2020-10-29 11:14:45,502 - WARNING - utils.load_model_from_checkpoint - Embedder or generator has changed in config, so not loading `optimizer_G`
PID 10490 - 2020-10-29 11:14:45,503 - INFO - train.py - Starting from iteration #2714183
PID 10490 - 2020-10-29 11:14:50,379 - WARNING - runner - Parameters mismatch in generator and the initial value of weights' running averages. Initializing by cloning
PID 10490 - 2020-10-29 11:14:50,386 - INFO - train.py - For fine-tuning, computing an averaged identity embedding from 409 frames
PID 10490 - 2020-10-29 11:14:53,639 - INFO - train.py - Entering training loop
/path/anaconda3/envs/latent-pose/lib/python3.7/site-packages/torch/nn/functional.py:3385: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
  warnings.warn("Default grid_sample and affine_grid behavior has changed "
Traceback (most recent call last):
  File "train.py", line 291, in <module>
    epoch, args, phase='train', writer=writer, saver=saver)
  File "/path/Documents/latent-pose-reenactment/runners/holycow.py", line 230, in run_epoch
    all_data_dict, losses_G_dict, losses_D_dict = training_module(data_dict, target_dict)
  File "/path/anaconda3/envs/latent-pose/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/path/latent-pose-reenactment/runners/holycow.py", line 178, in forward
    crit_out = criterion(data_dict)
  File "/path/anaconda3/envs/latent-pose/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/path/latent-pose-reenactment/criterions/dis_embed.py", line 22, in forward
    fake_embed = data_dict['embeds_elemwise']
KeyError: 'embeds_elemwise'

Hi, and thank you for letting me know. Indeed, I accidentally missed configs/ and Graphonomy because they were .gitignored.
Can you git pull and try again?

I pulled the latest code and fine-tuning works for me. However, I am not able to test segmentation by Graphonomy due to the unavailability of the universal trained model. I have opened an issue but did not get a response. Please share the universal trained model if that is okay for you.
I tested face reenactment but I am not able to reproduce the results as in the paper. I used the provided script for preprocessing. Even though my video is in 256x256, the script still adds the FFHQ-style reflection padding to the images-cropped results. When the cropped images have reflection padding, I got the following result. If I put the original images in the images-cropped folder, I got the following result.
Could you kindly provide image examples after preprocessing or detailed instructions on how to reproduce the resuls as in the paper?

It's a shame Graphonomy's link has expired. My bad I didn't check that, sorry. I've uploaded the checkpoint myself and have updated the instructions.
As for the preprocessing, the provided checkpoint is sensitive to the proper cropping of both identity sources and drivers! utils/preprocess_dataset.sh — or, more precisely, utils/crop_as_in_dataset.py — do just that, and they also do reflection padding if necessary, so yes, that padding is expected. In your example, the driver wasn't cropped, so make sure to follow all steps of the instructions and please report back if it worked.

I did reenactment with the provided Graphonomy checkpoint and got the following result. There seem to be segmentation or reflection padding artifacts in every frame. In the paper all figures except Figure 5 have no reflection padding.
For preprocessing, I used the following options.
DO_DECODE_VIDEOS=\
true

DO_CROP=\
true
DO_COMPUTE_SEGMENTATION=\
false
DO_COMPUTE_LANDMARKS=\
false
DO_COMPUTE_POSE_3DMM=\
false

DO_CROP_FFHQ=\
false
DO_COMPUTE_SEGMENTATION_FFHQ=\
false

For finetuning, I used 230 iterations for one identity image as suggested. I am not sure whether NUM_IMAGES=5 refers to how many images are there, the indexes of the images, or something else, so I just leave it as it is. The instructions mentioned underfitting or overfitting. I can see in the Tensorboard the losses are decreasing, but there are no ways for me to tell if that is a good fit or not. It would be great if can provide some rough numbers on how low the losses we should be expecting.

Please kindly let me know what else I may have missed in reproducing the results.

I did reenactment with the provided Graphonomy checkpoint and got the following result.
I can confirm that your result is valid, it looks reasonable.

There seem to be segmentation or reflection padding artifacts in every frame.
The reflection padding is expected if the face is cropped too tightly in the original images. If that happens for the driver, it's totally fine; if that happens for the identity, segmentation quality may suffer (like in your case) or at least the blurred reflection will be visible in the reenactment output.

To improve your particular result, you can get identity image(s) with a wider crop or of better quality, or in a pinch improve masks in $DATASET_ROOT/segmentation-cropped/$IDENTITY_NAME manually (via image editing software).

In the paper all figures except Figure 5 have no reflection padding.
Yes, because those images originally contained a larger region around the face. Though I have to admit that in this repository utils/preprocess_dataset.sh computes segmentation on already cropped (256x256) images. Such a "quick and dirty" solution is faster, but isn't optimal in terms of quality. As I said above, if you really need good segmentation masks, you should compute them on full images manually or edit them by hand. Or just feed in good quality identity images (usually the case with original VoxCeleb images), which is often enough too.

 Sorry for confusion. I've just improved the script so you don't have to set this manually anymore.

I can see in the Tensorboard the losses are decreasing, but there are no ways for me to tell if that is a good fit or not.
The "IMAGES" tab should be the most useful monitor for you. Open it and see when the identity gap is low enough. I've also improved the README on that.

Thank you for your detailed explanations. I selected this image as the identity, which should have enough space for cropping.
I finetuned for 230 and 460 iterations and below shows the tensorboard images.
These are the reenactment results for 230 and 460 iterations.
I observed the followings in the results.

The identity gap is still large.
The reflection paddings are still present.
The mouth shape is ambiguous.

What else would I have missed in reenactment?

The mouth shape is ambiguous.

Oh, I realized I made a silly error in the instructions, just corrected it. You should increase the number of iterations with the number of images. So, try less iterations, around 125.

The reflection paddings are still present.

You could use the driving video where there's originally more space around the face. Though I doubt this will improve something other than segmentation. Having more space in the identity image is more important.

The identity gap is still large.

There's not much I can suggest 🙁 There's a trade-off between identity gap and mimics. You could play with loss weights (like, increase facial identity loss' weight), try other images or videos, or try a better driver. No guarantees.

There's a trade-off between identity gap and mimics.

Could you shed light on why identity and pose have to be a trade-off but cannot be jointly optimized? Is it limited to only the architecture of this paper (latent-pose reenactment) or

Could you shed light on why identity and pose have to be a trade-off but cannot be jointly optimized?

Because we fine-tune to an extremely small dataset (as small as 1 image), so overfitting is almost inevitable and can be carefully handled.

the same limitation occurs also in similar meta-learning models such as few-shot talking head or bi-layer one-shot?

Few-shot talking heads: yes, since their principle is the same, and the architecture is very similar.
Bi-layer one-shot: not really, since they got rid of the fine-tuning stage; though the identity gap is of course still nonzero due to some new issues.