[BUG - Release] Game__Debug__Win64 does not successfully build

Describe the bug
GenericPlatformMath.hpp @ L410: error C2760: syntax error: 'UE_DEBUG_SECTION' was unexpected here; expected '{'
To Reproduce
Using my command line: xmake f --profilerFlavor=Tracy --cxxflags="-DSTATS=0" -m "Game__Debug__Win64" -y && xmake
Untested with the default command line or VS, but I'd be surprised if the problem didn't exist with those.
Expected behavior
The Game__Debug__Win64 target is expected to successfully build.
Desktop (please complete the following information):

OS: Win10 VM

I'm not sure how to properly fix this, but defining DO_ENSURE=0 bypasses the problem.

Perhaps fix could be bundled into #494?

Perhaps fix could be bundled into #494?
Edit: Build is successful on that branch already. I installed into DRG and it works 👍

Did you test #494 with the latest Unreal submodule ?
I'm pretty sure this problem was caused by a recent PR: https://github.com/Re-UE4SS/UEPseudo/pull/87 and #494 is based on an older commit so I just want to confirm that you tested with the latest Unreal submodule.

Perhaps fix could be bundled into #494?
Edit: Build is successful on that branch already. I installed into DRG and it works 👍

Did you test #494 with the latest Unreal submodule ? I'm pretty sure this problem was caused by a recent PR: Re-UE4SS/UEPseudo#87 and #494 is based on an older commit so I just want to confirm that you tested with the latest Unreal submodule.

Okay I merged #494's UEPseudo branch in with main locally and the debug build does indeed fail. Did not notice that #494 is based off an older commit.

Perhaps fix could be bundled into #494?
Edit: Build is successful on that branch already. I installed into DRG and it works 👍

Did you test #494 with the latest Unreal submodule ? I'm pretty sure this problem was caused by a recent PR: Re-UE4SS/UEPseudo#87 and #494 is based on an older commit so I just want to confirm that you tested with the latest Unreal submodule.

Okay I merged #494's UEPseudo branch in with main locally and the debug build does indeed fail. Did not notice that #494 is based off an older commit.

I can try to update and see if the change is something easily bundle-able. Will know in a couple of hours

#define UE_ENSURE_IMPL(Capture, Always, InExpression, ...) \
		    (LIKELY(!!(InExpression)) || (DispatchCheckVerify<bool>([Capture] () FORCENOINLINE UE_DEBUG_SECTION \
		    { \
			    static bool bExecuted = false; \
			    if ((!bExecuted || Always) && FPlatformMisc::IsEnsureAllowed()) \
			    { \
				    bExecuted = true; \
				    FDebug::OptionallyLogFormattedEnsureMessageReturningFalse(true, #InExpression, __FILE__, __LINE__, ##__VA_ARGS__); \
				    if (!FPlatformMisc::IsDebuggerPresent()) \
				    { \
					    FPlatformMisc::PromptForRemoteDebugging(true); \
					    return false; \
				    } \
				    return true; \
			    } \
			    return false; \
		    }) && ([] () { PLATFORM_BREAK(); } (), false)))

https://github.com/Re-UE4SS/UEPseudo/pull/87 introduced a breaking change whenever we compile in xxx__DEBUG__xxx mode due to a missing UE_DEBUG_SECTION def as well as unimplemented
FPlatformMisc:IsEnsureAllowed()
FPlatformMisc:IsDebuggerPresent()
FPlatformMisc::PromptForRemoteDebugging(bool)
I will not be making a fix in #494 since it's out of scope of that change.

https://github.com/Re-UE4SS/UEPseudo/pull/91


unknown identifier in ionic build --prod

Short description of the problem:
"ionic build android --prod" command fails with "Internal error: unknown identifier {}"
What behavior are you expecting?
Platform builds successfully
Steps to reproduce:

Create new project --v2
Update all packages
Run cmd ionic build android --prod

Which @ionic/app-scripts version are you using?
"dependencies": {
"@angular/common": "4.0.3",
"@angular/compiler": "4.0.3",
"@angular/compiler-cli": "4.0.3",
"@angular/core": "4.0.3",
"@angular/forms": "4.0.3",
"@angular/http": "4.0.3",
"@angular/platform-browser": "4.0.3",
"@angular/platform-browser-dynamic": "4.0.3",
"@ionic-native/camera": "3.5.0",
"@ionic-native/core": "3.5.0",
"@ionic-native/facebook": "3.5.0",
"@ionic-native/splash-screen": "3.5.0",
"@ionic-native/status-bar": "3.5.0",
"@ionic/storage": "2.0.1",
"ionic-angular": "3.0.1",
"ionicons": "3.0.0",
"ng-lazyload-image": "3.1.0",
"rxjs": "5.3.0",
"sw-toolbox": "3.6.0",
"zone.js": "0.8.8"
}
"devDependencies": {
"@ionic/app-scripts": "1.3.4",
"typescript": "~2.2.1"
}
Cordova CLI: 6.5.0
Ionic Framework Version: 3.0.1
Ionic CLI Version: 2.2.3
Ionic App Lib Version: 2.2.1
Ionic App Scripts Version: 1.3.4
ios-deploy version: Not installed
ios-sim version: Not installed
OS: Windows 10
Node Version: v7.8.0
Xcode version: Not installed
Other information: (e.g. stacktraces, related issues, suggestions how to fix, stackoverflow links, forum links, etc)
[12:44:31]  ionic-app-scripts 1.3.4
[12:44:31]  build prod started ...
[12:44:31]  clean started ...
[12:44:31]  clean finished in 3 ms
[12:44:31]  copy started ...
[12:44:31]  ngc started ...
[12:44:36]  ionic-app-script task: "build"
[12:44:36]  Error: Internal error: unknown identifier {}
Error: Internal error: unknown identifier {}
    at _TsEmitterVisitor._resolveStaticSymbol (\node_modules\@angular\compiler\bundles\compiler.umd.js:18944:19)
    at _TsEmitterVisitor._visitIdentifier (\node_modules\@angular\compiler\bundles\compiler.umd.js:18962:23)
    at _TsEmitterVisitor.visitExternalExpr (\node_modules\@angular\compiler\bundles\compiler.umd.js:18624:14)
    at ExternalExpr.visitExpression (\node_modules\@angular\compiler\bundles\compiler.umd.js:15817:24)
    at _TsEmitterVisitor.AbstractEmitterVisitor.visitBinaryOperatorExpr (\node_modules\@angular\compiler\bundles\compiler.umd.js:1
8315:17)
    at BinaryOperatorExpr.visitExpression (\node_modules\@angular\compiler\bundles\compiler.umd.js:15961:24)
    at _TsEmitterVisitor.AbstractEmitterVisitor.visitIfStmt (\node_modules\@angular\compiler\bundles\compiler.umd.js:18002:24)
    at IfStmt.visitStatement (\node_modules\@angular\compiler\bundles\compiler.umd.js:16384:24)
    at \node_modules\@angular\compiler\bundles\compiler.umd.js:18425:58
    at Array.forEach (native)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>ionic:build: `ionic-app-scripts build "--prod"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>ionic:build script 'ionic-app-scripts build "--prod"'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the ionic-politimob package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     ionic-app-scripts build "--prod"
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs ionic-politimob
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls ionic-politimob
npm ERR! There is likely additional logging output above.

I will close due to report in "ionic" repo


Question: 304

Response body is not read yet. Use SessionEventArgs.GetResponseBody() or SessionEventArgs.GetResponseBodyAsString() method to read the response body. at Titanium.Web.Proxy.Http.Response.EnsureBodyAvailable(Boolean throwWhenNotReadYet) at Titanium.Web.Proxy.EventArguments.SessionEventArgs.GetResponseBody(CancellationToken cancellationToken)
(I have no idea how to post a stack trace)
I've been getting this exception when trying to read the response body in the BeforeResponse handler. The exception makes no real sense because I'm always using GetResponseBody. What I did notice after a while is that this only happens on response status 304.
Is it possible that GetResponseBody is failing because of 304?

Yes, it is because of 304.... it has no body.

If it's not too much work, would it be possible to add a better exception for cases like this? Though this issue might be enough already. Thanks for the clarification!

Added: https://github.com/justcoding121/Titanium-Web-Proxy/commit/f2076892c676ba5e8ca1627e9984da1f75612b26

I close this issue.. feel free to reopen if needed.


Layer Switcher Control with Draw Plugin

When using the Draw plugin in combination with the Layer Switch control it would be handy to have the possibility to add the layer of drawn features as a layer or separated layers (one for each element? maybe too much).
In this way it would be possible to hide or show the feature, for instance when interacting with objects on the map eg:

need to perform a click for a popup on a layer below a drawn feature

The draw-plugin creates a lot of layers which we need to toggle.
It creates layers for every geometry type, active or inactive, ...
In the developer tools we can check it like this:
// Get all draw layer
_maplibreWidget._map.getLayersOrder().filter(l => l.startsWith("gl-draw"))

// Get all draw polygon layers
_maplibreWidget._map.getLayersOrder().filter(l => l.startsWith("gl-draw-polygon"))

_maplibreWidget._map.setLayoutProperty("gl-draw-polygon-fill-inactive.hot", "visibility", "none")
_maplibreWidget._map.setLayoutProperty("gl-draw-polygon-stroke-inactive.hot", "visibility", "none")

Therefore, I need to add layer groups that can be toggled.


How does jest-resolver treat ! in imports and requires?

I'm using script-loader for jquery plugins, and do a couple imports like this(in different places):
import 'script-loader!jquery-validation'
require('script-loader!jquery.maskedinput/src/jquery.maskedinput.js');

This is working at runtime on a browser, but not under Jest. I've tried adding script-loader.js to the  __mocks__  folder next to node_modules and confirm that my mock code gets run when I just do
jest.mock('script-loader')
require('script-loader')

in a test file.
I'm wondering if the ! in the imports are messing up the resolution? The exact error I'm getting is:
● Test suite failed to run

  Cannot find module 'script-loader!jquery.maskedinput/src/jquery.maskedinput.js' from 'index.tsx'

    at Resolver.resolveModule (C:/Users/kobyL/AppData/Roaming/npm/node_modules/jest/node_modules/jest-resolve/build/index.js:179:17)
    at Object.<anonymous> (src/components/MaskedInput/index.tsx:12:1)

Thanks!
Koby

+1

We appreciate Some samples. do you have one ?

Did you read the docs linked above?
If it doesn't work, please set up a small repo showing the error.

@SimenB using inline loader overrides is a valid use-case of webpack.. I think it would be amiss for jest not to support at least some way of working with them?
I tried adding the module to the moduleNameMapper jest config as "^(?:!!)?custom-loader!": "<rootDir>/__mocks__/customLoader.js" as I figured this would bypass trying to resolve the module, but jest is tripping up and erroring trying to resolve the file (I'm guessing this happens before honouring mocks?)

moduleNameMapper is for string => string, it does not support a module that does loading (if I understand your example correctly), it's basically aliasing. So you have to implement it as a regex replacement.
What's your use case for a loader in this case? Could it be implemented as a transform?
I still recommend sticking the loader in your webpack config to avoid tool-specific syntax in your code.
An alternative is to implement the loader using a babel plugin.

I was hoping that import '!!custom-loader!my-file' would get caught by "^!!custom-loader!": "<rootDir>/__mocks__/customLoader.js" and  never try to resolve !!custom-loader!my-file and instead just resolve to the mock? Just like mocking any other module or file?
My use-case is as follows:
I am using sass and have configured the sass/css loaders in my webpack.config. I have one occasion where I would like to import my main sass variables file into js using sass-variables-loader, for use in the JS code across the app. Hence I have a single instance where I need to override the normal webpack configuration and use the sass-variable-loader:
import vars from '!!sass-variable-loader!my-sass-vars.scss'

@Billy- I see, I'd think that was supposed to work... Can you try putting a breakpoint here and see what happens when your code runs? https://github.com/facebook/jest/blob/819cae5f18b6dc99c16ee2fc4e2df4a93b782290/packages/jest-resolve/src/index.js#L362-L398

I have one occasion where I would like to import my main sass variables file into js using sass-variables-loader

I still think it might be better to add that to exclude in your webpack config, or use oneOf and check for vars in the filename: https://webpack.js.org/configuration/module/#rule-oneof

Hmm, looks like it is hitting the mock now.. Must have been a typo or something cached. Sorry to waste your time, and thanks for the help!

@Billy- could you provide the working example? Is not clear how you've managed to make it work.

I solved this by adding the following to my jest config:
"moduleNameMapper": {
  "!file-loader!(.*)": "<rootDir>/$1"
}

hope this helps others!


TEP-0075: Add a pipeline run example with both object param and result

Changes
The pipeline consists of two tasks

task 1 writes the values of an object param to an object result.
task 2 uses task1's object result as the value of its own object param,
and echo that object param.

/kind misc

Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[ ] Has Docs included if any changes are user facing
[ ] Has Tests included if any functionality added or changed
[ ] Follows the commit message standard
[ ] Meets the Tekton contributor standards (including
functionality, content, code)
[ ] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
[ ] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
[ ] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes
Add a pipeline run example with both object param and result

/test check-pr-has-kind-label

/assign @ywluogg

Isn't https://github.com/tektoncd/pipeline/pull/5144/files a subset of these? Can they be combined?

Isn't https://github.com/tektoncd/pipeline/pull/5144/files a subset of these? Can they be combined?

sg! Just removed the old one since this new file covers more use cases.

/lgtm


Add Carthage support?

Could you please add Carthage support apart from SPM and Cocoapods?

What is needed for Carthage support? If I remember correctly, it should work out of the box, by just specifying the repository URL and target?

No, to support Carthage publisher should provide XCode Project with shared schema which can product XCFramework as build result.
Without this carthage will fail with Dependency "stinsen" has no shared framework schemes


Fix configure.py to properly compare "X.Y.Z" with "X.Y"

Right now, "0.24" is treated as lower than "..*" because of the odd comparison method that adds digits to each existing section. This change converts "0.24" to "0.24.0" to fix that.
This will probably need to be pulled into r2.0.
PiperOrigin-RevId: 241950768

@bananabowl @mihaimaruseac can we merge this ?

I think so. Added @goldiegadde to confirm

Can one of the admins verify this patch?


Related products - passing parent product ID

Hi guys, at the start I very appreciate this project. It's awesome!
In examples you show this code at views/woocommerce/single-product/related.twig:
<?php foreach ( $related_products as $related_product ) : ?>
    <?php
        $post_object = get_post( $related_product->get_id() );

        setup_postdata( $GLOBALS['post'] =& $post_object );

        wc_get_template_part( 'content', 'product' );
    ?>
<?php endforeach; ?>

Replaced with
{% for post in wc.related_products %}
    {{ fn('wc_get_template_part', 'content', 'product' ) }}
{% endfor %}

If you try it, in product detail it work's correct? If I look at related products, there are wrong data. Image, price and rating in all related products is inherited from main product.
After debugging loop displayed above I found that in file WooCommerce.php function maybe_render_twig_template_part stores in variable global $product currently displayed product, in global $post current related product's post. They are different, but code below passes to the context data based on global $product variable, which is currently displayed product.
if ( $product instanceof \WC_Product ) {
	$context['product'] = $product;
	$context['post_id'] = $product->get_id();
	$context['post']    = Timber::get_post( $product->get_id() );
}

If you can test it, please look at it, if it works correct.
Iny my case, I fixed it with getting product from current post, if global post and product ID's differ, for such cases:
// Add current product to context.
if ( $product instanceof \WC_Product ) {
	$context['product'] = $product;
	$context['post_id'] = $product->get_id();
	$context['post']    = Timber::get_post( $product->get_id() );
}
// MOD - Fix related.twig - there was passed parent product ID for related products
if($post->id !== $product->get_ID()){
	$context['product'] = wc_setup_product_data( $post );
	$context['post_id'] = $post->ID;
	$context['post']    = Timber::get_post( $post->ID );
}

Currently not tested at whole project, if are there some places where should be that ID's different.
Thank you for feedback.

Could you check whether you still run into that issue with the new version that is compatible with Timber v2?
https://github.com/mindkomm/timber-integration-woocommerce/releases/tag/1.0.0-rc.1

Closing this, because we didn’t get any feedback.


Cannot Extract Uploader ID

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.12.17
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

youtube-dl -f mp4 [debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://youtu.be/WWURxfAqPCI']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.10.9 (CPython) - Linux-6.2.1-zen1-1-zen-x86_64-with-glibc2.37
[debug] exe versions: ffmpeg 5.1.2, ffprobe 5.1.2, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] WWURxfAqPCI: Downloading webpage
ERROR: Unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
return func(self, *args, **kwargs)
File "/usr/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
ie_result = ie.extract(url)
File "/usr/lib/python3.10/site-packages/youtube_dl/extractor/common.py", line 534, in extract
ie_result = self._real_extract(url)
File "/usr/lib/python3.10/site-packages/youtube_dl/extractor/youtube.py", line 1794, in _real_extract
'uploader_id': self._search_regex(r'/(?:channel|user)/([^/?&#]+)', owner_profile_url, 'uploader id') if owner_profile_url else None,
File "/usr/lib/python3.10/site-packages/youtube_dl/extractor/common.py", line 1012, in _search_regex
raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sureto call youtube-dl with the --verbos

Description

Attempting to download a YouTube video causes an error in which the uploader ID could not be found. I have tried:

Specifying a format
Using verbose output to work out the issue
Using the -i flag
Using the --ignore-certificate-check flag
using -F
using -f
etc.
(Latest YTDL running on Arch installed through the AUR)

same problem

Same issue happing

This is fixed in the most recent version (not pushed out yet) if you clone the repo and run it form there you will be fine.

https://github.com/ytdl-org/youtube-dl/issues/31530

[x] I've searched the bugtracker for similar issues including closed ones

Read #30839 (again?) to learn how.


Changes to allow cylinders - and other cuboids - to go down

Hi Walter,
I was trying to create a new function like cylinder.js but that would create a hole in the ground. I traced cylinder --> arcImpl down to traverseHeight and noticed I could not pass a negative height in.
Would love some feedback on whether this approach makes sense or if you would do something different.
With this change I can now do
/js cylinder(blocks.air,5,-40)
to make a hole with radius 5 and 40 blocks deep
Best,
-Frank

Hey @kellyfj ,
If you want to build down, then why not move down first then build up?
/js down(40).cylinder(blocks.air,5,40)

hmmm. I think I see what you mean - you can do it by moving the drone first but using negative values for width, height and depth is more concise. I'll look at this some more.

I reckon if we go this route (negative distances) we'll have to do it for height, width and depth and update the docs accordingly. Will have a think about it.


Refactor reporter protocol

This is a large and invasive change, which does break one or two features, but it dramatically simplifies how fontbakery works internally.
Currently fontbakery checks yield a tuple to the check runner, which wraps it in another tuple (status, message, (section, check, iterargs)) to then yield on to reporters. Additionally, the checkrunner also yields various "pseudo-statuses" which are not check results - START, END, STARTCHECK, ENDCHECK and SECTIONSUMMARY - to "drive" the reporters. These overload the tuples in unexpected ways: for example, the "message" of the START status is the order of checks; the message of the END status is the counter of each status.
This PR starts in one direction - wrap these tuples in objects to clarify them - which helps to simplify the code, but out of that comes various realisations:

Multiprocessing dramatically complicates the check protocol; we can simplify it further by replacing it with multi-threading.
No reporter actually really cares about having a STARTCHECK, then each individual check result, then an ENDCHECK. Because we decide whether or not to report checks based on the highest status returned, we have to buffer all the check results until ENDCHECK anyway. So we can get rid of those and just return a single CheckResult wrapping the check invocation and all the subresults.
We don't need SECTIONSUMMARY or status counters in the checkrunner, since it's the job of reporters to tally statistics.
So actually we can just have the checkrunner call .start, .receive_result and .end methods on the reporters, and get rid of all the pseudo-statuses.
If a CheckResult knows how to describe itself in JSON then the JSON serialiser reporter becomes really trivial.

The features I have probably broken are -g (which seems to do different things in each reporter, so maybe that's not a great feature), -o (possibly? I'm not sure how it differs from -g), and I may have blurred the distinction between -m and -l, since that's not very clear to me either. But I think we're in a better place to put those back in now anyway. Also the multithreading sometimes works, but sometimes behaves weirdly...

Having the CheckResult and Subresult objects typed now shows that we have been playing fairly fast and loose with types up until now. I'll fix it.


Release Google.Cloud.Kms.V1 version 3.0.0

Changes in this release:
This is the first version of this package to depend on GAX v4.
There are some breaking changes, both in GAX v4 and in the generated code. The changes that aren't specific to any given API are described in the Google Cloud documentation. We don't anticipate any changes to most customer code, but please file a GitHub issue if you run into problems.
The most important change in this release is the use of the Grpc.Net.Client package for gRPC communication, instead of Grpc.Core. When using .NET Core 3.1 or .NET 5.0+ this should lead to a smaller installation footprint and greater compatibility (e.g. with Apple M1 chips). Any significant change in a core component comes with the risk of incompatibility, however - so again, please let us know if you encounter any issues.
API specific breaking changes

IAM access should now be via the IAMPolicyClient property in the client. (commit a47cb51)

Created release for Google.Cloud.Kms.V1-3.0.0


Fix a couple more ARIA issues.

Automated message from Dropbox CLA bot
@vaidap, it looks like you've already signed the Dropbox CLA. Thanks!

Merged, thanks @vaidap!


Move to folder based on date format and file timestamp

I'd love to have the ability to move to a folder based on a date format.  For example:
Folder: Meeting_Notes/YYYY-MM from tag: #meeting_notes
This would put files with that tag in the folder, but use the date format to create the folder and/or move the file into that folder making the folders dynamic, getting the YYYY-MM from the created date of the file or perhaps the filename with REGEX

Not exactly what you need but I'm doing something similar with my daily notes:
Date format: YYYY-MM-DD
Folder: Daily Notes Tag: Expression: [0-9]{4}-[0-9]{1,2}-[0-9]{1,2}
Would be nice if this program could also perform actions in addition to moving.

Also looking for a date based mover. I want to sort notes in to sub folders based on creation date


English Version

Hi,
I really love this admin panel and the UI in Vue. Do you have this in English version btw.
I see the locale setting in ii8n but something that can be defaulted to English version would help.
Thanks

I will try my best to realize the English version in the later stage.


Add support to read surpac strings

Need help creating a function to import surpac strings

Files structures for str and dtm are explained here:

https://www.cse.unr.edu/~fredh/papers/working/vr-mining/string.html


Add a metrics to track master RPC tput

What changes are proposed in this pull request?
This PR adds a new metric key to track master-side RPCs / sec
Why are the changes needed?
As requested from users, it is useful to know how busy the master is serving RPCs
Does this PR introduce any user facing changes?
Yes

@yuzhu Thank you for the summing the statistics suggestion, but It cannot be as same as this common rpc metrics, because we are not sure all of the individual rpc are computed to the aggregate result, from this common rpc path, it is easy to record the rpc throughput metrics.

Have an offline discussion with @apc999 , I'm ok for change the metrics implementation, as long as we supply the total rpc related metrics.

Let's make sure there is no impact to meta operations before checking this in.

@ggezer @apc999 Is there any update about meta operations impact? Hope to see this metrics.

@yuzhu  summarizing the metrics of total RPCs from individual RPC types is hard: these are timer with statistics like p50, p75, hard to aggregate to a new and total RPC metrics
@ggezer I created a new timer context to feed two timers a time. Let me know if you feel better in this way.
I feel we need a revamp of metrics counting after this PR. I will discuss with you two offline.

Codecov Report

Merging #13671 (28e4933) into master (0203366) will decrease coverage by 13.03%.
The diff coverage is 61.83%.

@@              Coverage Diff              @@
##             master   #13671       +/-   ##
=============================================
- Coverage     43.86%   30.82%   -13.04%
+ Complexity     8930     5816     -3114
=============================================
  Files          1359     1386       +27
  Lines         78109    80571     +2462
  Branches       9496     9803      +307
=============================================
- Hits          34260    24836     -9424
- Misses        40960    53640    +12680
+ Partials       2889     2095      -794

Impacted Files
Coverage Δ

...o/client/block/policy/DeterministicHashPolicy.java
68.42% <ø> (ø)

...nt/block/policy/LocalFirstAvoidEvictionPolicy.java
94.28% <ø> (ø)

.../alluxio/client/block/policy/LocalFirstPolicy.java
93.93% <ø> (ø)

.../client/block/policy/MostAvailableFirstPolicy.java
91.66% <ø> (ø)

.../alluxio/client/block/policy/RoundRobinPolicy.java
97.91% <ø> (ø)

...lluxio/client/block/policy/SpecificHostPolicy.java
40.00% <ø> (ø)

...xio/client/block/stream/BlockWorkerDataReader.java
0.00% <0.00%> (ø)

...ient/block/stream/BufferCachingGrpcDataReader.java
42.25% <0.00%> (+0.58%)
:arrow_up:

...main/java/alluxio/client/file/FileSystemUtils.java
0.00% <0.00%> (ø)

...va/alluxio/client/meta/MetaMasterConfigClient.java
0.00% <ø> (ø)

... and 659 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bfa1025...28e4933. Read the comment docs.

LGTM, That's what I want. Thank you

@maobaolong @ggezer  FYI:
@dbw9580  is helping using his RPC stressbench tool to verify if this PR introduces performance overhead.
Bowen, please post it here the results from RPC stressbench.
As discussed offline with @ggezer , if this PR is not introducing performance degradation, we will merge it

I've got some preliminary results.
I use the StressMasterBench to create 10000 test files at the master, and measure the throughput of doing the ListDir operation in a cluster of 1 master and 3 workers. Depending on how many simulated clients and how many threads each worker is using, the stress put on the master varies.
The command to start the benchmark looks like:
bin/alluxio runClass alluxio.stress.cli.StressMasterBench --operation ListDir --warmup 5s --duration 30s
 --clients 2 --threads 2 --fixed-count 10000 --cluster --cluster-limit 3 --target-throughput 100000
 --base 'alluxio://AlluxioSandbox-bowen-masters-1:19998/stress-master-base'

The result is below, where c is no. clients, and t is no. threads.
Throughput of ListDir:

parameters
c=1, t=1
c=2, t=2
c=4, t=4
c=1, t=10
c=10, t=10
c=10, t=20
c=10, t=50

baseline
17.739098
30.405294
35.441532
35.800377
35.580093
36.06496
38.521786

with metric added
18.597622
32.468178
37.721527
37.56974
37.519115
38.609425
39.19275

In every test the master with the metric added outperformed the baseline.  I'm pretty sure I didn't mistake one for another.
Really have no idea why it's like this... I thought it's due to a bug in the StressMasterBench, but even after I applied the fix for the baseline and this branch, and retested, the results remained the same.
More detailed results can be found here.

BTW, the metric is working as expected, although I think the time unit should be seconds instead of milliseconds.
$ curl -G http://AlluxioSandbox-bowen-masters-1:19999/metrics/json/ | jq '.timers["Master.TotalRpcs"]'
{
  "count": 217992,
  "max": 0.25309699999999996,
  "mean": 0.08051577074260588,
  "min": 0.002881,
  "p50": 0.022174,
  "p75": 0.14609999999999998,
  "p95": 0.162221,
  "p98": 0.16825199999999998,
  "p99": 0.17374799999999999,
  "p999": 0.25309699999999996,
  "stddev": 0.06746854157123174,
  "m15_rate": 9.854821419151405,
  "m1_rate": 6.241497317990331,
  "m5_rate": 6.325676776548836,
  "mean_rate": 53.71243696103727,
  "duration_units": "milliseconds",
  "rate_units": "calls/second"
}

More results in this spreadsheet: https://docs.google.com/spreadsheets/d/1ThgFXkYS_v7tryQkDudApeuW2hCwsC7D/edit?usp=sharing&ouid=108844305971006725786&rtpof=true&sd=true
I added this commit 29437ad3c53f8b648ee481d3a9cedb3229768a5a to simulate the old behaviour that calls a clock's getStep method to get the current time, instead of calling system.nanotime() directly. That's column withMetricEnabledUseClock in the spreadsheet.
The result is contrary to what I found yesterday: with this PR merged, the RPC throughput is somewhat reduced. Though the command I used is changed:
bin/alluxio runClass alluxio.stress.cli.StressMasterBench \
--operation ListDir --fixed-count 10000 \
--warmup 5s --duration 30s \
--cluster --cluster-limit 3 \
--target-throughput 100 \
--clients 8 --threads 64 \
--java-opt " -Xmx8g" \
--conf alluxio.user.file.writetype.default=MUST_CACHE \
--conf alluxio.user.block.master.client.pool.size.max=512 \
--conf alluxio.user.file.master.client.pool.size.max=512 \
--base 'alluxio://AlluxioSandboxbowen3-masters-1:19998/stress-master-base' >"listdir/run$i.json"

Notably no. clients is 8 and no. threads is 64. Also added a bunch of confs from a wrapper script that Overmind uses.

More results in this spreadsheet: https://docs.google.com/spreadsheets/d/1ThgFXkYS_v7tryQkDudApeuW2hCwsC7D/edit?usp=sharing&ouid=108844305971006725786&rtpof=true&sd=true
I added this commit 29437ad to simulate the old behaviour that calls a clock's getStep method to get the current time, instead of calling system.nanotime() directly. That's column withMetricEnabledUseClock in the spreadsheet.
The result is contrary to what I found yesterday: with this PR merged, the RPC throughput is somewhat reduced. Though the command I used is changed:
bin/alluxio runClass alluxio.stress.cli.StressMasterBench \
--operation ListDir --fixed-count 10000 \
--warmup 5s --duration 30s \
--cluster --cluster-limit 3 \
--target-throughput 100 \
--clients 8 --threads 64 \
--java-opt " -Xmx8g" \
--conf alluxio.user.file.writetype.default=MUST_CACHE \
--conf alluxio.user.block.master.client.pool.size.max=512 \
--conf alluxio.user.file.master.client.pool.size.max=512 \
--base 'alluxio://AlluxioSandboxbowen3-masters-1:19998/stress-master-base' >"listdir/run$i.json"

Notably no. clients is 8 and no. threads is 64. Also added a bunch of confs from a wrapper script that Overmind uses.

Thanks @dbw9580. Could you please retry this after-before test with this same command and share how much throughput change you're seeing?

Thanks @dbw9580. Could you please retry this after-before test with this same command and share how much throughput change you're seeing?

@ggezer
The results from running this command with and without this PR merged are summarized in this spreadsheet, and the raw output JSON files generated by StressMasterBench are in this folder.
GetFileStatus and GetBlockLocation saw -2.5% and -3.8% performance impact, respectively, compared to the baseline which is this commit. For ListDir the performance penalty is up to -13.8%. To be extra sure about this, I ran ListDir tests twice in a new cluster and got an identical result.
I also added a new commit that replaced the direct usage of System.nanotime() with a call to clock.getTick(), which is what it used to be without this PR. To my surprise, doing so seems to improve the performance a bit (compared to using System.nanotime() directly).

Did some micro benchmarks with this commit.
I find that with this PR merged, on my machine it adds a constant ~200ns on each invocation to callAndReturn, regardless of how long the RPC call actually takes. For example, calling callAndReturn with a no-op dummy RPC call takes ~500ns on baseline, and ~700ns with this pr merged. Increase the delay, the time goes up but the difference remains the same. In other words, the longer the rpc call takes, the more negligible impact this PR would have on performance.

Did some micro benchmarks with this commit.
I find that with this PR merged, on my machine it adds a constant ~200ns on each invocation to callAndReturn, regardless of how long the RPC call actually takes. For example, calling callAndReturn with a no-op dummy RPC call takes ~500ns on baseline, and ~700ns with this pr merged. Increase the delay, the time goes up but the difference remains the same. In other words, the longer the rpc call takes, the more negligible impact this PR would have on performance.

I've got more reliable results with the microbenchmark tools in https://github.com/Alluxio/alluxio/pull/14003.
Details are in this summary spreadsheet: https://docs.google.com/spreadsheets/d/1ThgFXkYS_v7tryQkDudApeuW2hCwsC7D/edit#gid=1377711729
Conclusions:

the PR introduce a constant ~200ns additional delay.
the relative performance impact dwindles as the rpc takes longer time.
when the rpc takes <1us to finish, the relative performance impact is -10.5%.
when the rpc takes 9us to finish, the relative performance impact is -2%.

@ggezer  PTAL

alluxio-bot, merge this please

@apc999 Wow, I wait for this great metric for a long time, very excited it merged. Thanks @apc999 and @ggezer @dbw9580 @yuzhu


One-Click Hide Giveaway Button (2.37) not showing on group's pages

Description
The "eye" button from 2.37 does not show up on group's giveaway pages. (Though neither does steamgifts eye to hide all giveaways for a specific game.)
On the whitelist page I'm getting the tool-tip of the feature when mouse over on top of the eye but that eye is still steamgifts eye to remove all giveaways for that game. Not just this specific giveaway.
Console Errors
none
System (please complete the following information):

ESGST Version 8.3.3
Chrome Version 73.0.3683.86 (Official Build) (64-Bit)

Additional Context
It's odd,... I was told I was expected to see two eyes when activating the feature - the one from from SG (to hide all GAs of a game) and another one from ESGST (to just hide a specific GA). I don't get the latter though.

Are you confusing features? One-Click Hide Giveaway Button simply replaces SG' button. To hide specific giveaways, it's Giveaway Filters > Single Filters.

Thanks, that was it.
As saying, I was just going by what I was told and I couldn't find any other thing that fit the description I got.
Now I'm just wondering if there's a way/page where I can see all the hidden single giveaways to possibly un-hide single ones again?
I'm asking because I've hidden a specific giveaway on the Wishlist page but the button (present/crossed out eye) that should be on the page heading as per feature description can nowhere be found on the page.


Deserialisation of PathBuf doesn't work

Deserialising a simple struct with a PathBuf inside fails with the error: SyntaxError
Can be reproduced by this Gist:
https://gist.github.com/neon64/03230cb40ff3bafed8ee#file-bincode_pathbuf-rs
#![feature(plugin)]
#![feature(custom_derive)]
#![plugin(serde_macros)]

extern crate serde;
extern crate bincode;

use std::path;
use bincode::SizeLimit;
use bincode::serde::{serialize, deserialize};

#[derive(Serialize, Deserialize)]
struct MyData {
    path: path::PathBuf
}

#[test]
fn it_works() {
    let data = MyData {
        path: path::Path::new("foo").to_path_buf()
    };
    let ser = serialize(&data, SizeLimit::Infinite).unwrap();
    let de: MyData = deserialize(&ser).expect("bincode failed");
}

It seems like it is just limited to bincode, as serde_json works for example.

The main issue is that Bincode was built for rustc-serialize, and there are some serde idioms that don't translate well.
The SyntaxError (updated to show useful information in c778cda42fc013c95e91b58ade2d235fe4475899) is hitting one of the things that bincode can't do.  Namely that Bincode does not support Deserializer::visit.
However, PathBuf probably shouldn't require the kind of hinting that Deserializer::visit uses.
Resolutions:

I'll open up a bug on Serde and see if I can get this fixed on their side.
For now, you can just encode / decode using Strings.  Sorry about the inconvenience.

This was fixed upstream in serde!


Improve scheduler event messages

Scheduler event:
Events:
  FirstSeen  ... Reason                    Message
  ...              ...  failedScheduling	Error scheduling: For each of these fitness predicates, pod frontend failed on at least one node: PodFitsResources.

Problems with this event message:

It says Error, but having pods fail to schedule is a common event in a system, and "Error" is too alarming a term.
It says scheduling, but the event makes it clear that this is about scheduling in both the From and Reason fields.
it says For each of these fitness predicates .. failed on at least one node: but there is only one predicate in this case.  Lets be concise and to the point where possible.  Maybe when only one predicate failed, say: No node satisfied rule PodFitsResources.
the message contains the string pod frontend, but that information is already present in the event object.  Save space by removing this.  Also, it reduces the cardinality of the set of distinct message strings, which facilitates grouping of events.

An improved output looks something like this:
Events:
  FirstSeen  ... Reason                    Message
  ...              ...  failedScheduling	No node found where PodFitsResources.

I'm going to close this issue and copy the comments over to #9207.


YAML files in the crds folder of a Helm v3 template are interpreted as YAML and not as helm-template

Original issue
As of now, the templates folder apparently gets recognized and therefore YAML files in it get the helm-template language mode assigned. This should also be enabled for the crds folder introduced in Helm v3.

Thanks for letting us know @theAkito. We'll give it a look.

@bacongobbler would you be able to look at this please?

@bacongobbler would you be able to look at this please?

This seems correct. YAML files inside the crds/ directory cannot be templated, therefore they should be treated as plain YAML files.

This seems correct. YAML files inside the crds/ directory cannot be templated, therefore they should be treated as plain YAML files.

From https://helm.sh/docs/topics/charts/#custom-resource-definitions-crds:

CRD files cannot be templated. They must be plain YAML documents.

From https://helm.sh/docs/topics/charts/#custom-resource-definitions-crds:

CRD files cannot be templated. They must be plain YAML documents.


can't access pod logs after some event

It's not clear to me what causes this, it seems to happen on the two systems that have produced the most log output but there isn't a clear event when it happens. Neither system is anywhere near capacity, though both are heavily loaded.
I0528 19:57:06.729518   29212 loader.go:359] Config loaded from file /snap/microk8s/526/client.config
I0528 19:57:06.730077   29212 round_trippers.go:419] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.13.5 (linux/amd64) kubernetes/2166946" 'http://<IP_ADDRESS>:8080/api/v1/namespaces/default/pods/ace-service-55b6457685-rlt49/log?container=ace-service&follow=true'
I0528 19:57:06.733367   29212 round_trippers.go:438] GET http://<IP_ADDRESS>:8080/api/v1/namespaces/default/pods/ace-service-55b6457685-rlt49/log?container=ace-service&follow=true 404 Not Found in 3 milliseconds
I0528 19:57:06.733395   29212 round_trippers.go:444] Response Headers:
I0528 19:57:06.733406   29212 round_trippers.go:447]     Content-Type: application/json
I0528 19:57:06.733413   29212 round_trippers.go:447]     Date: Tue, 28 May 2019 19:57:06 GMT
I0528 19:57:06.733423   29212 round_trippers.go:447]     Content-Length: 270
I0528 19:57:06.733480   29212 request.go:942] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource ( pods/log ace-service-55b6457685-rlt49)","reason":"NotFound","details":{"name":"ace-service-55b6457685-rlt49","kind":"pods/log"},"code":404}
I0528 19:57:06.733873   29212 helpers.go:198] server response object: [{
  "metadata": {},
  "status": "Failure",
  "message": "the server could not find the requested resource ( pods/log ace-service-55b6457685-rlt49)",
  "reason": "NotFound",
  "details": {
    "name": "ace-service-55b6457685-rlt49",
    "kind": "pods/log"
  },
  "code": 404
}]
F0528 19:57:06.733905   29212 helpers.go:116] Error from server (NotFound): the server could not find the requested resource ( pods/log ace-service-55b6457685-rlt49)

Neither restarting microk8s nor rebooting the server seems to fix it, but after taking one of them out of commission over the weekend, a reboot did fix it. So it seems like something to do with either load or log rotation or something.
This is microk8s 1.13/stable as the move to 1.14 broke the system in a bunch of other ways, so upgrading is less of an option to me.
Docker logs for the container still work, as do tailing the files in the file system, but any access via kubectl fails with the above NotFound.
I've not found other reports of this after googling for either kubernetes or microk8s specifically (nor run into it while administering 100 node k8s clusters), so I feel like it is more likely to be on the microk8s side.
inspection-report.tar.gz

I see the same on v1.18.4. The cluster has been running under heavy-ish load for about 4 months, but today it suddenly stopped accepting logs requests:
I0630 09:31:09.055384   16168 helpers.go:216] server response object: [{
  "metadata": {},
  "status": "Failure",
  "message": "the server could not find the requested resource ( pods/log postgres-postgresql-0)",
  "reason": "NotFound",
  "details": {
    "name": "postgres-postgresql-0",
    "kind": "pods/log"
  },
  "code": 404
}]
F0630 09:31:09.055516   16168 helpers.go:115] Error from server (NotFound): the server could not find the requested resource ( pods/log postgres-postgresql-0)

I have tried microk8s.stop/microk8s.start without success. Most other commands and communication between pods seem to work.

The problem might (seems to, but possibly a coincidence) have started with an automatic snapd update:
Jun 30 08:34:39 stfv-hpcpro02 snapd: AppArmor status: apparmor not enabled
Jun 30 08:34:39 stfv-hpcpro02 snapd: patch.go:64: Patching system state level 6 to sublevel 1...
Jun 30 08:34:39 stfv-hpcpro02 snapd: patch.go:64: Patching system state level 6 to sublevel 2...
Jun 30 08:34:39 stfv-hpcpro02 snapd: patch.go:64: Patching system state level 6 to sublevel 3...
Jun 30 08:34:39 stfv-hpcpro02 snapd: daemon.go:343: started snapd/2.45-1.el7 (series 16; classic; devmode) rhel/7.8 (amd64) linux/3.10.0-1127.10.1.el7.x86_.
Jun 30 08:34:39 stfv-hpcpro02 snapd: daemon.go:436: adjusting startup timeout by 40s (pessimistic estimate of 30s plus 5s per snap)
Jun 30 08:35:04 stfv-hpcpro02 microk8s.daemon-apiserver-kicker: /var/lib/snapd/snap/microk8s/1341/apiservice-kicker: /snap/microk8s/1341/bin/sleep: /snap/core/current/lib64/ld-linux-x86-64.so.2: bad ELF interpreter: No such file or direc
tory
Jun 30 08:35:05 stfv-hpcpro02 snapd: daemon.go:542: gracefully waiting for running hooks
Jun 30 08:35:05 stfv-hpcpro02 snapd: daemon.go:544: done waiting for running hooks
Jun 30 08:35:05 stfv-hpcpro02 snapd: AppArmor status: apparmor not enabled
Jun 30 08:35:05 stfv-hpcpro02 snapd: daemon.go:343: started snapd/2.45-1.el7 (series 16; classic; devmode) rhel/7.8 (amd64) linux/3.10.0-1127.10.1.el7.x86_.
Jun 30 08:35:05 stfv-hpcpro02 snapd: daemon.go:436: adjusting startup timeout by 40s (pessimistic estimate of 30s plus 5s per snap)
Jun 30 08:35:16 stfv-hpcpro02 microk8s.daemon-flanneld: I0630 08:35:16.098848    1588 main.go:437] Stopped monitoring lease
Jun 30 08:35:16 stfv-hpcpro02 microk8s.daemon-flanneld: I0630 08:35:16.101448    1588 main.go:339] Waiting for all goroutines to exit
Jun 30 08:35:16 stfv-hpcpro02 microk8s.daemon-flanneld: I0630 08:35:16.104236    1588 main.go:370] shutdownHandler sent cancel signal...
Jun 30 08:35:16 stfv-hpcpro02 microk8s.daemon-flanneld: I0630 08:35:16.113436    1588 main.go:342] Exiting cleanly...

Then a lot of noise in the logs for several minutes (all microk8s services, sandboxes, pods restarted). Many of which are errors, but I don't know if they mean anything or are just transient. After everything had stabilized, this error keeps getting reported in the logs:
Jun 30 09:56:30 stfv-hpcpro02 microk8s.daemon-etcd: WARNING: 2020/06/30 09:56:30 grpc: addrConn.createTransport failed to connect to {<IP_ADDRESS>:12379 0  <nil>}. Err :connection error: desc = "transport: Error while dialing failed to do connect handshake, response: \"HTTP/1.1 503 Service Unavailable\\r\\nConnection: close\\r\\nContent-Length: 768\\r\\nCache-Control: no-cache\\r\\nContent-Type: text/html; charset=utf-8\\r\\nPragma: no-cache\\r\\nProxy-Connection: close\\r\\n\\r\\n\\r\\n<TITLE>Network Error</TITLE>\\r\\n\\r\\n\\r\\n\\r\\n<big></big>
\\r\\n\\r\\n
\\r\\n
\\r\\n
\\r\\n\\r\\n<big>Network Error (tcp_error)</big>\\r\\n
\\r\\n
\\r\\n\\r\\n
\\r\\n
\\r\\n\\r\\nA communication error occurred: \\\"\\\"\\r\\n\\r\\n
\\r\\n
\\r\\n\\r\\nThe Web Server may be down, too busy, or experiencing other problems preventing it from responding to requests. You may wish to try again at a later time.\\r\\n\\r\\n
\\r\\n
\\r\\n\\r\\n
\\r\\nFor assistance, contact your network support team.\\r\\n\\r\\n
\\r\\n
\\r\\n
\\r\\n\\r\\n\\r\\n\"". Reconnecting...

I can provide the full logs from after the snapd upgrade on request, but it's ~1900 lines on system I can't easily copy files out from, so only if interesting.

Actually I recognize that 503 message. It's from the system's $http_proxy server, which shouldn't be used here of course.
That log message is not logged in the previous 4 weeks, so whatever causes it was triggered by the snapd upgrade.

(Side note: I wasn't aware of the auto refresh policy of snap, and I don't like it. Although I wouldn't call it real production, the services I run under microk8s have users and is meant to be stable and up at all times - unless I decide an upgrade is warranted, and notify the users. I will try to enable the "app awareness" mode of snap to see if it helps to stop the automatic refreshes. https://forum.snapcraft.io/t/wip-refresh-app-awareness/10736)

I've had this same problem for some time (Feb/March 2020). At some point I presume a SNAP update broke something, the since then I've not been able to correct this. The observation is this:
$ microk8s kubectl -n dse logs component-5d95b44657-zwmmn -c adapter
Error from server (NotFound): the server could not find the requested resource ( pods/log component-5d95b44657-zwmmn)

But of course the logs are being produced (note I need sudo to read them):
$ sudo cat /var/log/pods/dse_component-5d95b44657-zwmmn_e40b8253-8398-4721-b77a-3d
ba12f9101b/adapter/0.log
2020-09-28T11:15:47.620748345Z stderr F INFO:dse.component.component:Component:
2020-09-28T11:15:47.62076961Z stderr F INFO:dse.component.component:    uid=1234

I wonder if anyone as an idea how to recover from this?

@trulede , @termie - do you have HTTP_PROXY/HTTPS_PROXY set? I suspect it may be relevant for my problem somehow, but if it's only me then it's probably a red herring.

Yes, they are set, however if I prefix the logs command with unset ....  to remove that, it does not help.

I think this may be related, https://github.com/etcd-io/etcd/issues/9763.
Unsetting the environment variables in the logs command will not help, it's some cluster-internal communication that fails.
I don't know how to fix it (possibly by setting --advertise-client-urls in /var/snap/microk8s/current/args/etcd and restarting etcd, in combination with a no_proxy environment variable, or perhaps just setting no_proxy=localhost in /etc/environment). But I can't experiment with my "problematic" system now, because it's in use.


Dynamic page layout handling

Page Layout service will be able to filter page layout by optional PageLayoutHandler. The PageLayoutHandler will be able to modify the template given by the backend (or static config).
One use case for this is the dynamic building of the cart page; while the cms provides statically a paragraph in case there's no cart entries, we cannot always show this. The PageLayoutHandler would be able to change this based on the cart entries.

QA steps:

Check if cart page doen't display empty cart message when cart is not empty, and display cart items instead
Check if cart displays empty cart message when cart is empty

Test successful. Cart status displayed correctly according empty/non-empty status 🆗


Error

It seems there's an issue with the JSON file. Please make sure you have filled in the token and staff role ID, as well as any other necessary role IDs, properly. Additionally, there might be more role IDs to be pasted in the main.py file; make sure you have walked through it. I have already mentioned it as a comment. Don't forget to go to the last line of main.py to set the token :)

Fixed the issue


Progress made with Lazarus/Free Pascal + Native Debug

lldb --version: lldb-370.0.42 -
Hello, further to my post a couple of months ago https://github.com/WebFreak001/code-debug/issues/106 , I have tried again with Native Pascal, and have managed to get further this time.
I'm not seeing the flurry of error messages in the debug console so that's good. I am, however, still having problems accessing variables. As you can see from the screenshot below, I have, for example, a variable called Code. This is an instance of the TCodeBuffer class. The default entry in the variable list doesn't work because Lazarus/FPC does a dereferencing trick to make the true value of *Code appear as Code in the code. If however, I add a watch with the variable specified as *CODE then I get to see the variables class name displayed. It also successfully displays simple class members, such as strings, numbers and booleans.
The second screen shot shows the Xcode application working with LLDB to debug the same Lazarus/FPC application. My logic is that if Xcode can do it, then it must be theoretically possible for Native Debug to do the same thing. I'm not suggesting it's easy, just theoretically possible!

Why not just use Xcode, you may ask. Well, VSCode is a much more pleasant application to use compare to Xcode, IMO.
I know you said you don't have any FPC/Lazarus/Pascal experience but hopefully, I've been able to supply a little more useful information this time.
To recap then, can you help me uncover more of LLDB's hidden variable information?
Thanks in advance,
Carl

can you send me some source that is as minimal as possible to replicate the problem?

Hi Jan, thanks for the reply.
I’ve uploaded a sample app to https://dropfile.to/mpZpcbp access code is D51wjvO.
It’s a simple console app with a simplified but similar structure to the app I’m working on at the moment.
Try putting a breakpoint in the Pascal source file uwebfreak.pas
Cheers,
Carl
On 26 Jul 2017, 19:39 +0100, Jan Jurzitza<EMAIL_ADDRESS>wrote:

can you send me some source that is as minimal as possible to replicate the problem?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Could you provide me with build instructions?
I tried lazbuild webfreak.lpi but it errored:
Error: (lazarus) invalid Lazarus directory "": directory lcl not found
Error: (lazarus) Building failed: webfreak.lpi

Give me a few minutes. I built it from within Lazarus itself with no special parameters, but I’ll try and see what are the correct parameters for using Lazbuild...
On 26 Jul 2017, 21:09 +0100, Jan Jurzitza<EMAIL_ADDRESS>wrote:

Could you provide me with build instructions?
I tried lazbuild webfreak.lpi but it errored:
Error: (lazarus) invalid Lazarus directory "": directory lcl not found
Error: (lazarus) Building failed: webfreak.lpi
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I do have the following environment variables set:
Lazarus stuff
export PP=/usr/local/bin/fpc
export FPCDIR=/usr/local/share/fpcsrc
export LAZARUSDIR=/Developer/lazarus
export FPCTARGET=i386-darwin
export FPCTARGETCPU=i386
The paths are based on standard default installations of Lazarus.
I don’t know if Lazbuild uses them or not...
On 26 Jul 2017, 21:16 +0100, Carl Caulkett<EMAIL_ADDRESS>wrote:

Give me a few minutes. I built it from within Lazarus itself with no special parameters, but I’ll try and see what are the correct parameters for using Lazbuild...
On 26 Jul 2017, 21:09 +0100, Jan Jurzitza<EMAIL_ADDRESS>wrote:

Could you provide me with build instructions?
I tried lazbuild webfreak.lpi but it errored:
Error: (lazarus) invalid Lazarus directory "": directory lcl not found
Error: (lazarus) Building failed: webfreak.lpi
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Do you have the full Lazarus installed or just Free Pascal?
On 26 Jul 2017, 21:24 +0100, Carl Caulkett<EMAIL_ADDRESS>wrote:

I do have the following environment variables set:
Lazarus stuff
export PP=/usr/local/bin/fpc
export FPCDIR=/usr/local/share/fpcsrc
export LAZARUSDIR=/Developer/lazarus
export FPCTARGET=i386-darwin
export FPCTARGETCPU=i386
The paths are based on standard default installations of Lazarus.
I don’t know if Lazbuild uses them or not...
On 26 Jul 2017, 21:16 +0100, Carl Caulkett<EMAIL_ADDRESS>wrote:

Give me a few minutes. I built it from within Lazarus itself with no special parameters, but I’ll try and see what are the correct parameters for using Lazbuild...
On 26 Jul 2017, 21:09 +0100, Jan Jurzitza<EMAIL_ADDRESS>wrote:

Could you provide me with build instructions?
I tried lazbuild webfreak.lpi but it errored:
Error: (lazarus) invalid Lazarus directory "": directory lcl not found
Error: (lazarus) Building failed: webfreak.lpi
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

both, are you sure the lpi file is correct? It references <PackageName Value="LCL"/> and stuff

That lip should have compiled, but I have removed the lcl dependency.
Try this…
https://dropfile.to/EjVxCW9 access key: fSNzkRg
On 26 Jul 2017, 21:36 +0100, Jan Jurzitza<EMAIL_ADDRESS>wrote:

both, are you sure the lpi file is correct? It references  and stuff
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I’m sorry to mess you around. I do appreciate the effort you are taking over this!
On 26 Jul 2017, 21:44 +0100, Carl Caulkett<EMAIL_ADDRESS>wrote:

That lip should have compiled, but I have removed the lcl dependency.
Try this…
https://dropfile.to/EjVxCW9 access key: fSNzkRg
On 26 Jul 2017, 21:36 +0100, Jan Jurzitza<EMAIL_ADDRESS>wrote:

both, are you sure the lpi file is correct? It references  and stuff
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

what's your build procedure, I haven't setup anything yet, I just installed lazarus and fpc from the arch repositories

Did you start Lazarus so far? AFAIK it's doing some configuration on first start up.

When you set the breakpoint in LLDB, did you try doing `print *LIST' ? Like I said, Lazarus and Delphi do some dereferencing tricks to make variables which should appear as * actually appear as . LLDB doesn't understand this, so you have to speak in real "pointer" terms to it.

Hi Jan, if you have a moment, grab the latest version of the sample app at https://dropfile.to/cYpBezD access code Evzwk2x. Stick a break point in LLDB at line 69, then do 'p *OBJ.LIST`. You should see the contents of the List object which is a child property of the Obj object. In other words, LLDB does allow the display of child properties. Interestingly, I've just tried this via the Add Watch facility in VSCode and it seems to work here. It didn't seem to work in the actual app I'm working on, though.
So what would be good is if the list of Variables was able to detect which of the variables was a class reference, and to ask LLDB for the * value, so that we don't have to add the values manually as watches.
I hope that all makes sense.
Cheers,
Carl

Jan, something bad has happened! I no longer am able to stop at breakpoints at all. I've tried uninstalling and reinstalling Native Debug, but I cannot get breakpoints to work at all. This rather blows my progress out of the water. Do you have any idea what the problem might be?

have you tried using a different compiler or different arguments to emit gdb compatible debug symbols?

No, I haven't done anything differently, as far as I know. It just stopped working suddenly.

It hasn't worked since last night. Something has happened overnight to cause me to lose the use of breakpoints.

It's OK my LLDB is working again. Not sure what the problem was, to be honest.

Any thoughts about the P *OBJ.LIST thing I mentioned earlier?

I was trying to get LLDB to work with Free Pascal and found this thread. On my system (10.13 lldb-900.0.45) the "p" command is broken and crashing but "fr v" works and displays members variables of classes.
Look at this example and notice how "p" crashes at the end but I get member variables using fr v. Not sure if this is relevant or helps.
(lldb) fr v MODEL
(TMODEL) MODEL = 0xbffff95c
(lldb) fr v *MODEL
(TMODEL) *MODEL = {
  TOBJECT = {
    REGISTEREDMETHODS = 0x000007c3
    MANAGEDOBJECTS = 0x0064bf00
    AUTORELEASECOUNT = 0
    RETAINCOUNT = 0
    DIDINITIALIZE = true
    INITIALIZEDFROMCOPYING = true
    COPYPARAMETERS = 0xb9938c00
  }
  MESH = (VERTEXATTRIBS = 0x3e45a200, VERTEXSIZE = -1476722686, V = 0x00000000, IND = 0xe04e6620)
  MATERIAL = {
    SHINEDAMPER = 0.961168169
    REFLECTIVITY = -5.949044E+19
    TEXTURES = (M_DATA = 0x3f760f1e, M_ACTUALSIZE = 19562528, M_GROWSIZE = -1073743432, DATA = 0x3f760f1e)
    COLOR = (X = 0.0000000000000000000000000000000000000000971225952, Y = 0.000000000000000000000000000000000000000100446475, Z = 0.00000000000000000000000000000000000000000112103877, R = 0.0000000000000000000000000000000000000000971225952, G = 0.000000000000000000000000000000000000000100446475, B = 0.00000000000000000000000000000000000000000112103877)
    ID = -1073743368
  }
  SHADER = 0x0001190c
  ID = 1599792
  VERTEXBUFFERID = 1599792
  INDEXARRAYBUFFERID =<PHONE_NUMBER>
  VAO = 213793
}
(lldb) fr v *MODEL->SHADER
(TSHADER) *MODEL->SHADER = {
  TOBJECT = {
    REGISTEREDMETHODS = 0x8b042474
    MANAGEDOBJECTS = 0x8d08247c
    AUTORELEASECOUNT = 9316
    RETAINCOUNT = 49932
    DIDINITIALIZE = true
    INITIALIZEDFROMCOPYING = true
    COPYPARAMETERS = 0xf424648d
  }
  PROGRAMID =<PHONE_NUMBER>
  VERTEXSHADERID =<PHONE_NUMBER>
  FRAGMENTSHADERID =<PHONE_NUMBER>
  UNIFORMS = 0x89241c89
}
(lldb) p *MODEL
Illegal instruction: 4

@carlca @genericptr Any update on this? Could you possibly retry that, maybe with an updated LLDB and also with GDB?


ResponseEntity Support

Hello,
I have a client like this that I'm trying to make it work using reactive feign.
@ReactiveFeignClient(name = "mktcloud-service", url = "http://localhost:8092")
public interface JourneyV1Client {
  @RequestMapping(
      value = "/v1/journeys/apikey",
      produces = {"application/json"},
      method = RequestMethod.GET)
  Mono<ResponseEntity<ApiKeySearchResponseDTO>> findApiKey(
      @NotNull
          @ApiParam(value = "Journey Name", required = true)
          @Valid
          @RequestParam(value = "name", required = true)
          String name);
}

However when I run the application I face the following issue:
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.springframework.http.ResponseEntity` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (io.netty.buffer.ByteBufInputStream); line: 1, column: 2]
	at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67) ~[jackson-databind-2.10.2.jar:2.10.2]
	at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1589) ~[jackson-databind-2.10.2.jar:2.10.2]

On regular feign there is the Encoder and Decoder concept, and spring implementation of openfeign uses a ResponseEntityDecoder to wrap the other decoders.
I did not see a similar way to perform this on reactive feign, so I think this must be an issue, as Spring support seems to be part of the project.

Hi, @gmcouto .
Why do you need this?
BTW: There is no commitment to support all Spring features. The basic goal of this project to help developers write effective and clean code.

It is mainly because we are using OpenAPI-generator to generate all the interfaces on an API-first application as part of the build process.
As the interface generator for spling-cloud includes ResponseEntity, and it is supported by OpenFeign, I thought it would be useful to have it here... so the client can use the same API interface we use for controllers.

Here is the example

Thanks, sorry for not answering before.
I was thinking on something like this:
https://github.com/gmcouto/feign-reactive/commit/42211aa95efaa95d23087dc83abebaead019c65f
So it is transparent to the application. That is the version we are using to have this support.
Can I open a Merge Request?

It

Actually the interface openapi is generator is generating this return type:
Mono<ResponseEntity<ApiKeySearchResponseDTO>>
Which was causing the errors described in the original post. The PR works for it, which is the fix we are using in our codebase.

It's not reactive. Reactive version is Mono<ResponseEntity<Mono/Flux>>

Can you customize openapi-generator to generate API in form Mono<ReactiveHttpResponse<Mono<ApiKeySearchResponseDTO>>>  ?

Why is that not reactive? It is an implementation of WebClientUtils, uses WebClient for the request (supposed to be reactive), returns a Mono, and no single .block() is called in the process.
Sorry, I`m new to reactive in Java so I may be missing something.

Mono<ReactiveHttpResponse<Mono<ApiKeySearchResponseDTO>>> allows you to subscribe on response, read headers and drop body if needed. In your case body will be always deserialized.
Mono<ReactiveHttpResponse<Flux<ApiKeySearchResponseDTO>>> allows to subscribe to result and process flux of elements. And if you need omit rest of elements at some points. In your case all elements will be deserialized. I think that spring should fix their implementation to support ResponseEntity<Mono/Flux<R>> .

I get it now, and that is a good point.
I think however, for the exception of lists, anyone using feign does intend to deserialize the whole body all the time. Which is what I have changed, and is working for us.
Anyway, I will investigate if it is too much on an effort to make our controllers work with ReactiveHttpResponse and change open-api, because, ideally, I wanted to use the same interface in both sides(controller/client) auto-generated, for consistency. It is beginning to seem like a luxury though.

As I see, ideally to ask Spring to support Mono<ResponseEntity<Mono<ApiKeySearchResponseDTO>>>

I have even prepared the fix on reactive feign side but it fails when I call toEntity(ParameterizedTypeReference<Mono<ApiKeySearchResponseDTO>>)

fixed
here is the example Mono<ResponseEntity<Flux>> call()

Hello @kptfh , I think when you say

ideally to ask Spring to support Mono<ResponseEntity<Mono>>

you actually mean https://github.com/OpenAPITools/openapi-generator instead of Spring ;)
I've added some comments into this, backing your arguments to support it in that way: https://github.com/OpenAPITools/openapi-generator/issues/1309#issuecomment-673163349
Cheers!

I have something like this: Mono<ResponseEntity<Flux>>
The way I am processing the request is something as follows:
val response = serviceClient.request().awaitFirstOrNul()
if(response.isSuccessful()) {
val invoices = response.body.collectList()?.awaitFirstOrNull()
}
but the following exception is being rasied
java.lang.IllegalStateException: Only one connection receive subscriber allowed.


erorr line in resize image using fit in visual studio code

I work with Visual Studio Code 1.46.1 and Ci 4.0.2. For resize image I have this code:
$image = \Config\Services::image()
        ->withFile('/path/to/image/mypic.jpg')
        ->fit(100, 100, 'center')
        ->save('/path/to/image/mypic_thumb.jpg');

this work but i see red line error in visual studio code like this:

when i change fit to crop red line is removed and work true. i think fit function have a problem.

I can confirm this also with latest develop branch. No problem with code. Problem however is on PHPDoc block of fit showing return type of boolean instead of $this.

I'll make a PR later.


allow for no XML to be provided

this is necessary for smog-server files to be used in opensmog.  There may be other cases where no XML file is needed.

The "noxml" flag has been added to loadSystem.
see commit 2a43ac7
I'm leaving this open until I had some new checks to opensmogcheck.


(OraklNode) Update gas limit

Description
Gas limit increase after unix milli precision update
failed tx example:
https://klaytnscope.com/tx/0x68d9731189f55c459f0b772703fb6a9c42734a48b7d6f30b8f0cc8c60fcaf544?tabId=inputData
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist before requesting a review

[ ] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.

Deployment

[ ] Should publish npm package
[ ] Should publish Docker image

will close the pr since it seemed to be a temporary issue


Velocity struct does not implement Bevy's Component trait

Hello,
I recently updated to the latest version of Bevy (0.12.0) and encountered an issue with the Velocity struct from bevy_rapier2d. The Velocity struct does not implement Bevy's Component trait, which is now required for all components in Bevy's ECS (Entity Component System).
Here's the error message I'm getting:
error[E0277]: the trait bound `bevy_rapier2d::dynamics::Velocity: bevy::prelude::Component` is not satisfied
   --> src/game/asteroids_system.rs:137:16
    |
137 |     asteroids: Query<(&Asteroid, &Transform, &Velocity)>,
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `bevy::prelude::Component` is not implemented for `bevy_rapier2d::dynamics::Velocity`

(By the way your work is great. Thank you for your time and for your contributions to this project.)

This is because bevy_rapier has not (yet) been updated for Bevy 0.12, and Rust treats traits from different versions of a crate as different (for reasons outside the scope of this, and that honestly I don't fully know/understand myself). #439 seems to be ready to merge, so we're just waiting on that to happen and then a new version of bevy_rapier to be released.
(I have no affiliation with this project, I'm just some random guy who happens to also be using it and it also awaiting it to support Bevy 0.12 so I can upgrade.)

Thank you @Kromey I will keep an eye on the update release

@rubenVillalvazo Should be released now.

Thank you @Aceeri I'm going to update it right now

I believe that's fixed now! Closing.


Problems when using wechaty as token gateway server

I'm running wechaty docker version as token gateway server on Ubuntu 18.04 (Let's say this one is the Server)
I'm running wechaty python version , with the last example code. (Let's say this one is the Client)
There are 2 problems here.

Every time Client will show 2 different QR codes, both can be scanned and log wechat successfully.

After logged into wechaty, I tried to send "ding" to my wechat bot. Server shows error with

messagePattern: 'Assertion failed'

At the same time. Client ran into endless loop, keeps printing
2020-05-28 08:50:33 INFO wechaty.py <error_listener> receive <error> event <(<Status.INTERNAL: 13>, 'Assertion failed', None)>

in the console.

Here is the verbose log which may help to locate the problem.
08:40:11 VERB EventStreamManager grpcEmit(EVENT_TYPE_SCAN[22], {"qrcode":"http://weixin.qq.com/x/YYe...first_code","status":2}) 08:40:11 VERB PuppetServiceImpl ding() 08:40:11 VERB EventStreamManager grpcEmit(EVENT_TYPE_DONG[3], {"data":"ding-dong"}) 08:40:12 VERB EventStreamManager grpcEmit(EVENT_TYPE_HEARTBEAT[1], {"data":"","timeout":60000}) 08:40:12 VERB PadplusManager setContactAndRoomData() can not proceed due to no cache. 08:40:15 INFO IoClient [2] http://weixin.qq.com/x/IYl2i....second_code 08:40:15 VERB EventStreamManager grpcEmit(EVENT_TYPE_SCAN[22], {"qrcode":"http://weixin.qq.com/x/IYl2i....second_code","status":2}) 08:40:15 VERB PuppetServiceImpl ding() 08:40:15 VERB EventStreamManager grpcEmit(EVENT_TYPE_DONG[3], {"data":"ding-dong"}) 08:40:29 VERB EventStreamManager grpcEmit(EVENT_TYPE_HEARTBEAT[1], {"data":"","timeout":60000}) 08:40:29 VERB PadplusManager setContactAndRoomData() can not proceed due to no cache.

The 2nd one:
08:49:28 VERB Puppet messagePayload(4548687293285196463)
Error [AssertionError]: Assertion failed
    at new goog.asserts.AssertionError (/wechaty/node_modules/google-protobuf/google-protobuf.js:81:876)
    at Object.goog.asserts.doAssertFailure_ (/wechaty/node_modules/google-protobuf/google-protobuf.js:82:257)
    at Object.goog.asserts.assert [as assert] (/wechaty/node_modules/google-protobuf/google-protobuf.js:83:83)
    at jspb.BinaryEncoder.writeUnsignedVarint64 (/wechaty/node_modules/google-protobuf/google-protobuf.js:458:77)
    at jspb.BinaryWriter.writeUnsignedVarint64_ (/wechaty/node_modules/google-protobuf/google-protobuf.js:480:507)
    at jspb.BinaryWriter.writeUint64 (/wechaty/node_modules/google-protobuf/google-protobuf.js:485:471)
    at Function.proto.wechaty.puppet.MessagePayloadResponse.serializeBinaryToWriter (/wechaty/node_modules/@chatie/grpc/dist/generated/wechaty/puppet/message_pb.js:849:12)
    at proto.wechaty.puppet.MessagePayloadResponse.serializeBinary (/wechaty/node_modules/@chatie/grpc/dist/generated/wechaty/puppet/message_pb.js:812:47)
    at serialize_wechaty_puppet_MessagePayloadResponse (/wechaty/node_modules/@chatie/grpc/dist/generated/wechaty/puppet_grpc_pb.js:460:26)
    at sendUnaryResponse (/wechaty/node_modules/grpc/src/server.js:98:15)
    at sendUnaryData (/wechaty/node_modules/grpc/src/server.js:600:9)
    at Object.<anonymous> (/wechaty/node_modules/wechaty-puppet-hostie/src/server/puppet-implementation.ts:524:16)
    at Generator.next (<anonymous>)
    at fulfilled (/wechaty/node_modules/wechaty-puppet-hostie/dist/src/server/puppet-implementation.js:5:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  reportErrorToServer: true,
  messagePattern: 'Assertion failed',
  code: 13
}

Following steps may help you to reproduce the problem.

Clone the latest version of python-wechaty-getting-started
Run docker version wechaty as token gateway server.
Install requirements, set environments.
Run python example code
The program shows more than one QRCode at this moment.
Scan one of the codes and log into wechaty successfully.
Use another wechat account send a message to this one.
The Server got Assertion failed, Client ran into endless loop

Thanks for the description for the reproduce steps!
May I ask you that what kind of the token are you using for the gate service? Padplus, or puppeteer?

Thanks for the description for the reproduce steps!
May I ask you that what kind of the token are you using for the gate service? Padplus, or puppeteer?

I’m using padplus token.

@zavierxu  you can try
docker run -e "WECHATY_PUPPET_PADPLUS_TOKEN=puppet_padplus_xxxxxxxxxxx"
    docker.pkg.github.com/wechaty/scala-wechaty/ding-dong-bot:0.0.16-SNAPSHOT

@zavierxu Please update the Wechaty Docker Image to the latest for fix this problem with the Token Gateway.


darwin.builder: Example flake throws both null and not null error on system.nixos.revision

Describe the bug
I tried to incorporate the flake usage example from the docs on darwin.builder into my flake-based macOS configuration. It fails with
error: The option `system.nixos.revision` is defined both null and not null, in `<nix-store-path-to-nixpkgs>/flake.nix` and `<nix-store-path-to-nixpkgs>/nixos/modules/profiles/macos-builder.nix`.

Assuming it's a problem with my nix configuration, I tried darwin-rebuild build --flake .#machine1 directly on the flake usage example from the docs. Again, I get the same error. Am I wrong in assuming that this should at least build the example flake? Any pointers to what I'm doing wrong?
Steps To Reproduce

Create new directory with flake.nix from flake usage example.
Run darwin-rebuild build --flake .#machine1 in that directory.
Build fails with above error message.

Expected behavior
I would assume that the build goes through without any error messages.
Notify maintainers
@Gabriella439
Metadata
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-darwin"`
 - host os: `Darwin 22.4.0, macOS 10.16`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.15.0`
 - channels(peter): `""`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

cc: @G3zz since they contributed the "Example flake usage" section

Didn't see this until now - sorry.!
The problem is that system.nixos.revision is set to null in macos-builder.nix, but the nixos flake defining nixosSystem sees a rev defined and tries to set the system.nixos.revision to that rev. The solution is to forcibly set the revision to null, I think:
{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-22.11-darwin";
    darwin.url = "github:lnl7/nix-darwin/master";
    darwin.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, darwin, nixpkgs, ... }@inputs:
  let

    inherit (darwin.lib) darwinSystem;
    system = "aarch64-darwin";
    pkgs = nixpkgs.legacyPackages."${system}";
    linuxSystem = builtins.replaceStrings [ "darwin" ] [ "linux" ] system;

    darwin-builder = nixpkgs.lib.nixosSystem {
      system = linuxSystem;
      modules = [
        "${nixpkgs}/nixos/modules/profiles/macos-builder.nix"
        { virtualisation.host.pkgs = pkgs;
          system.nixos.revision = nixpkgs.lib.mkForce null;
        }
      ];
    };
  in {

    darwinConfigurations = {
      machine1 = darwinSystem {
        inherit system;
        modules = [
          {
            nix.distributedBuilds = true;
            nix.buildMachines = [{
              hostName = "ssh://builder@localhost";
              system = linuxSystem;
              maxJobs = 4;
              supportedFeatures = [ "kvm" "benchmark" "big-parallel" ];
            }];

            launchd.daemons.darwin-builder = {
              command = "${darwin-builder.config.system.build.macos-builder-installer}/bin/create-builder";
              serviceConfig = {
                KeepAlive = true;
                RunAtLoad = true;
                StandardOutPath = "/var/log/darwin-builder.log";
                StandardErrorPath = "/var/log/darwin-builder.log";
              };
            };
          }
        ];
      };
    };

  };
}

Hi, I'm also trying this for the first time, so please bear with me if the question is stupid, but I'm using almost the same config in the documentation, including the mkForce null for the revision, and I'm finding this error:
error: a 'aarch64-linux' with features {} is required to build '/nix/store/z2j4abagl1vmpfyddwx9qghics1adk4d-append-initrd-secrets.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}

Is this related to the supportedFeatures attribute in nix.buildMachines? How could I know more about this error so I can fix it?
In case you want to check, I'm trying to use the builder from this darwinSystem: https://github.com/DavSanchez/nix-dotfiles/blob/bc03e0c18b807ed6a21bbb6714ec369a476ecb72/flake.nix#L141

Hi, I'm also trying this for the first time, so please bear with me if the question is stupid, but I'm using almost the same config in the documentation, including the mkForce null for the revision, and I'm finding this error:
error: a 'aarch64-linux' with features {} is required to build '/nix/store/z2j4abagl1vmpfyddwx9qghics1adk4d-append-initrd-secrets.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}

Make sure you are setting
virtualisation.host.pkgs = nixpkgs.legacyPackages.aarch64-darwin;

Hi, I'm also trying this for the first time, so please bear with me if the question is stupid, but I'm using almost the same config in the documentation, including the mkForce null for the revision, and I'm finding this error:

error: a 'aarch64-linux' with features {} is required to build '/nix/store/z2j4abagl1vmpfyddwx9qghics1adk4d-append-initrd-secrets.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}

Make sure you are setting

virtualisation.host.pkgs = nixpkgs.legacyPackages.aarch64-darwin;

I think I am (check line 12): https://github.com/DavSanchez/nix-dotfiles/blob/feat/nixos-vms-on-darwin/hosts/darwin-builder.nix

I think I am (check line 12): https://github.com/DavSanchez/nix-dotfiles/blob/feat/nixos-vms-on-darwin/hosts/darwin-builder.nix

Yeah, story checks out.
See https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder

Thanks! I have made some progress, but looks like I'm still missing something.
Sorry for hijacking this thread, please feel free to point me to another place if continuing here is not appropriate.
I follow what is documented here:

I do nix run nixpkgs#darwin.linux-builder
I add the lines to nix.conf (note this is a symlink from previous darwin-rebuilds that set options there, but I overwrite it for the bootstrapping)
Create /etc/ssh/ssh_config.d/100-linux-builder.conf with the same contents so I can find the machine
I restart the Nix daemon
Then I run the darwin-rebuild ... from my flake and it succeeds

At this point, can I quit the builder that I set up at step 1 and run the flake as long as the launchd service is enabled? There's org.nixos.darwin-builder running when I do launchctl list, but if I try to create a VM having quitted the machine created in step 1 it fails again with:
$ nix run '.#darwinVM'
warning: Git tree '/Users/david/.dotfiles' is dirty
cannot build on 'ssh://ssh://builder@localhost': error: failed to start SSH connection to '//ssh://
builder@localhost': ssh: connect to host localhost port 22: Connection refused
Failed to find a machine for remote build!
derivation: 5q9bymmyqffl17wcdfk8hgixdvjlxn8p-ensure-all-wrappers-paths-exist.drv
required (system, features): (aarch64-linux, [])
1 available machines:
(systems, maxjobs, supportedFeatures, mandatoryFeatures)
([aarch64-linux], 4, [benchmark, big-parallel, kvm], [])
Failed to find a machine for remote build!
derivation: 90d8hpx9qqm2nyv2xfbww0z9k6d9x9x7-users-groups.json.drv
required (system, features): (aarch64-linux, [])
1 available machines:
(systems, maxjobs, supportedFeatures, mandatoryFeatures)
([aarch64-linux], 4, [benchmark, big-parallel, kvm], [])
error: a 'aarch64-linux' with features {} is required to build '/nix/store/5q9bymmyqffl17wcdfk8hgix
dvjlxn8p-ensure-all-wrappers-paths-exist.drv', but I am a 'aarch64-darwin' with features {benchmark
, big-parallel, nixos-test}

I notice the ssh://ssh:// at the second line and try to edit the launchd service by removing the protocol: hostName = "builder@localhost"; on nix.buildMachines. I rebuild successfully, but I still cannot reach the machine:
nix run '.#darwinVM'
warning: Git tree '/Users/david/.dotfiles' is dirty
cannot build on 'ssh://builder@localhost': error: failed to start SSH connection to 'builder@localh
ost': ssh: connect to host localhost port 22: Connection refused
Failed to find a machine for remote build!
derivation: 5q9bymmyqffl17wcdfk8hgixdvjlxn8p-ensure-all-wrappers-paths-exist.drv
required (system, features): (aarch64-linux, [])
1 available machines:
(systems, maxjobs, supportedFeatures, mandatoryFeatures)
([aarch64-linux], 4, [benchmark, big-parallel, kvm], [])
Failed to find a machine for remote build!
derivation: 90d8hpx9qqm2nyv2xfbww0z9k6d9x9x7-users-groups.json.drv
required (system, features): (aarch64-linux, [])
1 available machines:
(systems, maxjobs, supportedFeatures, mandatoryFeatures)
([aarch64-linux], 4, [benchmark, big-parallel, kvm], [])
error: a 'aarch64-linux' with features {} is required to build '/nix/store/5q9bymmyqffl17wcdfk8hgix
dvjlxn8p-ensure-all-wrappers-paths-exist.drv', but I am a 'aarch64-darwin' with features {benchmark
, big-parallel, nixos-test}

What am I missing here?

@DavSanchez Had the same problems this troubleshooting helped a lot, thx!
Additionally, my builder was not starting probably, investigating the logs /var/log/darwin-builder.log I discovered:
qemu-img: /nixos.qcow2: error while converting qcow2: Could not create '/nixos.qcow2': Read-only file system
mkdir: cannot create directory './keys': Read-only file system
Saving key "./keys/builder_ed25519" failed: No such file or directory
cmp: ./keys/builder_ed25519.pub: No such file or directory

changing the serviceConfig WorkingDirectory helped:
  launchd.daemons.darwin-builder = lib.mkIf enableDarwinBuilder {
    command = "${darwin-builder.config.system.build.macos-builder-installer}/bin/create-builder";
    serviceConfig = {
      KeepAlive = true;
      RunAtLoad = true;
      StandardOutPath = "/var/log/darwin-builder.log";
      StandardErrorPath = "/var/log/darwin-builder.log";
      WorkingDirectory = "/etc/nix/";
    };
  };

Now with PID:
sudo launchctl list | grep darwin-builder
59337   0       org.nixos.darwin-builder

My current working setup: https://github.com/MaxDaten/.home/blob/50afcbf9f1d567c9be821237bf70fd26fce9d252/machines/macos/modules/builder.nix#L50

https://github.com/NixOS/nixpkgs/pull/248554 implements the suggested fix from @MaxDaten

Thanks all! I followed @MaxDaten indication and got it to work.

Still doesn't work for me: error: a 'x86_64-linux' with features {} is required to build '/nix/store/pvbyf17zrrasf48wncdkgbfjd3pplbpv-append-initrd-secrets.drv', but I am a 'x86_64-darwin' with features {benchmark, big-parallel, nixos-test}

I was also running into this issue until I realized my (very dumb) mistake: the imperative commands shown in the manual are not an alternative to the declarative configuration, but required to bootstrap them (@sikmir, perhaps this is also your issue?). Thanks @DavSanchez  and @Gabriella439 for your explanations!
In case it helps anyone, this is the module I use to load the virtual machine.
{
  inputs,
  system,
  pkgs,
  lib,
  ...
}: let
  linuxSystem = builtins.replaceStrings ["darwin"] ["linux"] system;

  darwin-builder = lib.nixosSystem {
    system = linuxSystem;
    modules = [
      "${inputs.nixpkgs}/nixos/modules/profiles/macos-builder.nix"
      {
        virtualisation.host.pkgs = pkgs;
        system.nixos.revision = lib.mkForce null;
      }
    ];
  };
in {
  nix.distributedBuilds = true;
  nix.buildMachines = [
    {
      hostName = "localhost";
      sshUser = "builder";
      publicHostKey = "c3NoLWVkMjU1MTkgQUFBQUMzTnphQzFsWkRJMU5URTVBQUFBSUpCV2N4Yi9CbGFxdDFhdU90RStGOFFVV3JVb3RpQzVxQkorVXVFV2RWQ2Igcm9vdEBuaXhvcwo=";
      system = linuxSystem;
      maxJobs = 4;
      supportedFeatures = ["kvm" "benchmark" "big-parallel"];
      protocol = "ssh-ng";
    }
  ];

  launchd.daemons.darwin-builder = {
    command = "${darwin-builder.config.system.build.macos-builder-installer}/bin/create-builder";
    serviceConfig = {
      KeepAlive = true;
      RunAtLoad = true;
      StandardOutPath = "/var/log/darwin-builder.log";
      StandardErrorPath = "/var/log/darwin-builder.log";
      WorkingDirectory = "/etc/nix/";
    };
  };
}

Hopefully someone can learn from my mistakes :)

the imperative commands shown in the manual are not an alternative to the declarative configuration, but required to bootstrap them

@Samuel-Martineau I have encountered this issue, and I guess I can learn from your answer. What do you mean by bootstrap them? I tried to use your config, and run:
nix run nix-darwin -- --flake .#machine1 switch

But still it does not work, showing this error mesage:
error: a 'aarch64-linux' with features {} is required to build '/nix/store/7177gzz2sxb0n7xzrx0rndzi776l0a3d-X-Restart-Triggers.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}

@winston0410 The idea is that you need a Linux builder to build the Linux VM. To overcome this, you can use the prebuilt Linux VM from the nix run command above. Run this in an other shell, add it to your /etc/nix/nix.conf and then build the Darwin generation. Once the build is done, you can stop the VM started imperatively.


[SDK] Add read-only access to the StateStore

Components (Tasks) other than the scheduler may wish to understand the state of a given service without having access to modifying service state.  The addition of a read-only StateStore interfaces allows for this.
Example uses include:

Cassandra seed provider
Cross-service locality

This PR looks like it has many changes when in fact it is a pure lift of methods from StateStore to StateStoreReadOnly in both interfaces and implementation.

retest this please

retest this please

The code seems fine, but rather than deepen the hierarchy now rooted at StateStoreReadOnly, what I'd propose is to take a page from the Java collections book and retain just a single interface -- StateStore -- with different implementors for the read-only and read/write cases (and maybe, in deference to the Rule of Least Power, with names that make mutability sound like the exceptional case, e.g. CuratorStateStore and MutableCuratorStateStore), and where the read-only variant throws java.lang.UnsupportedOperationException for any write calls. This retains the same level of code reuse (the mutable concrete implementation can extend the read-only implementation) with the only downside being that the compiler won't catch some contract violations until runtime, but I foresee that being rare in practice and worth the simplification of the type hierarchy here.

I'd lean towards the type-aware enforcement, but how about this structure?:

@nickbp: Thought about the casting approach, but wanted to enforce things such that one couldn't escape from readonly mode to writable mode.  I guess this isn't that big a deal.  If a developer wants to break stuff with code, there are easier ways to do it.
@mrbrowning: I agree with @nickbp, I prefer types.  Throwing exceptions on unsupported operations seems weird.  Why expose what can't be used?

I think the nature of Java's type system is such that all of the available options have glaring drawbacks. We know the drawbacks of throwing in unsupported methods, but as for the approach here: inheritance implies that the subclass satisfies at least all of the invariants of the superclass; so we can pass a mutable StateStore to any site expecting a read-only StateStore. The response is, of course, "don't mutate the state store in read methods," but that's exactly the same logic underlying "don't call write methods on a read-only state store" except that the invariant is broken silently, in a much less obvious way, and maybe much later! So as far as this dilemma goes, the type system is mostly pageantry no matter how you slice it in terms of what behavior it guarantees. I'd much rather runtime checks that fail aggressively and loudly than weak contracts that are violated behind closed doors.

Having two separate state store implementations separately accessing the same underlying storage will make caching semantics more complicated. At the moment there's a thread-safe singleton StateStoreCache wrapping a singleton CuratorStateStore.


Render components on server too for SSR

We probably should avoid using the /client folder?

awesome :)

Actually, this works as far as not triggering the error, but the component is still not rendered server-side. I don't know how easy it would be to actually do this though, as some of Meteor's own APIs are only available client-side…

I don't think anything is really gained by having the login completely render on the server and would add to the complexity.  Most SSR is for search engine cache or to make the page render before everything is ready... Neither really apply for login.

Maybe you're right. It's just visually jarring to have the log in button pop in after a delay when you first load the site. Maybe as a compromise only the button could be server-side rendered?

OK changed it to actually render the component server side in 1.1.0


New app router errors

With new app routing navigating by url  dosent work and give error.

It partially works if I change like this, but still give an error,

Does anybody how to fix it?

@alexei-petru thanks for pointing this out. Looking into a fix!

Thank you for fast fix, you are the best.
There are two more warnings, should I ignore them? Does anybody know why and how to fix them? I have searched for a solution and could not find one.

I noticed these as well, but they did seem like just warnings and not actually affecting anything. If you see otherwise, please let me know! I'll keep trying to figure out a better long term solution as well. It's pretty strange in its current state 😅


(SERVER-220) Make jetty a regular dependency

@cprice404 thoughts?

@jasperla Thanks for the submission! See #355 for the follow up and https://tickets.puppetlabs.com/browse/SERVER-220 for tracking progress.
Closing in favor of #355

@nwolfe thanks for the follow-up.


LNK Path

Implement path printing for LNK files. Details are under [MS-SHLLINK].pdf.

Apparently this was done back in 2c288b4258f037052797020d59977b03f06eb05b, but I'll add a convenient wprintf-like function to support other non-Windows systems.


Best chain BlockId not working properly in a parachain context

In a parachain, the runtime is embedded in a "Cumulus" runtime. What happens is the Block in the embedded runtime is yet to be aware of the best_chain's BlockId passed from the frontier::rpc. Used this way throws arbitrary errors when calling client.runtime_api():
let header = self.select_chain.best_chain();
client.runtime_api().account_basic(&BlockId::Number(header.number()), address);

Results in:
UnknownBlock(
    "Require header: BlockId::Number(XX)",
)

If I artificially add a sleep delay (i.e. 2 seconds) after the best_chain call in frontier::rpc and before the actual client.runtime_api call, then it works fine:
let header = self.select_chain.best_chain();
thread::sleep(X);
client.runtime_api().account_basic(&BlockId::Number(header.number()), address);

Closing as we will track this in https://github.com/PureStake/moonbeam/issues/15. Will re-open here if necessary.


RFC use develop branch for next changes

Hey @webmozart ,
I was looking for a way to handle json schema $ref locally and was happy that your package provides a nice mechanism: https://github.com/webmozart/json#schemas
We have the latest version installed 1.2.2 but the JsonValidator looks different in this version. So I was confused. Wouldn't it be better to work on upcoming feature in a develop branch so that the README in the master branch reflects the latest released version instead of the next?

I'm closing this. We now have the same discussion at prooph https://github.com/prooph/proophessor/issues/41 and maybe github-flow wins ;)


error in class name imported in visualize_dataset.py

dgp/dgp/scripts/visualize_dataset.py attempts to SynchronizedDataset from dgp.datasets.synchronized_dataset, when the class is actually named _SynchronizedDataset

Hello Sidnuro, we've updated the visualization code. Could you try this CLI again?


Remove the loading indicator or activity indicator

I am using the configuration below, however, it is not fully invisible. The activity indicator appears on iOS. The StatusBar appears on Android.
I am using the following configuration. modalProps prop not rewrites modalProps props correctly.
                    <View style={{ display: 'none' }}>
                        <Recaptcha
                            style={{ display: 'none' }}
                            ref={recaptcha}
                            siteKey="6LcsO5sUAAAAADhfEx8vydV0wfzsU2nP1DjHjaZN"
                            baseUrl="https://www.trollflix.com"
                            onVerify={onVerify}
                            size="invisible"
                            hideBadge={true}
                            modalProps={{ visible: false, animationType: 'none', transparent: true, supportedOrientations: ['landscape-right'], statusBarTranslucent: true }}
                        />
                    </View>

You can change de loading component with loadingComponent prop, but I don't think that is a good option to completely remove it.
Because reCaptcha can open de challenge popup in the front of your screen, like this:

About the status bar on Android, you can change it with statusBarTranslucent in modalProps https://reactnative.dev/docs/modal#statusbartranslucent-android


Support home switch inputs

Add input with configurable polarity for home switch.
Add a go-home command, that takes as a parameter a homing-speed.
This should go with constant speed in the direction specified by signed velocity, until home switch is triggered. At that point set the current position (via offset or otherwise) to 0, without breaking commutation.

Task List

[ ] Add hardware support
[ ] Add callbacks
[ ] Homing speed limit
[ ] Homing function
[ ] Fault on endstop pressed flag

Merged into RC & devel


Move B+tree into namespace and remove using namespace from any headers

Move B+tree into namespace and remove using namespace from any headers.

Codecov Report

Merging #131 (a870321) into main (d9a352d) will increase coverage by 0.00%.
The diff coverage is 100.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@           Coverage Diff           @@
##             main     #131   +/-   ##
=======================================
  Coverage   98.32%   98.32%
=======================================
  Files          26       26
  Lines        2084     2085    +1
=======================================
+ Hits         2049     2050    +1
  Misses         35       35

Impacted Files
Coverage Δ

include/phtree/common/b_plus_tree_hash_map.h
98.38% <ø> (ø)

include/phtree/common/b_plus_tree_map.h
98.13% <ø> (ø)

include/phtree/common/common.h
100.00% <ø> (ø)

include/phtree/common/flat_array_map.h
99.02% <ø> (ø)

include/phtree/common/flat_sparse_map.h
97.91% <ø> (ø)

include/phtree/phtree.h
100.00% <ø> (ø)

include/phtree/phtree_multimap.h
99.50% <ø> (ø)

include/phtree/v16/iterator_full.h
100.00% <ø> (ø)

include/phtree/v16/iterator_lower_bound.h
98.50% <ø> (ø)

include/phtree/filter.h
100.00% <100.00%> (ø)

... and 7 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.