(IMPORT FAILED) Conflicted Nodes:ComfyUI-Inference-Core-Nodes

(IMPORT FAILED) ComfyUI's ControlNet Auxiliary Preprocessors	This is a rework of comfyui_controlnet_preprocessors based on ControlNet auxiliary models by 🤗. I think the old repo isn't good enough to maintain. All old workflow will still be work with this repo but the version option won't do anything. Almost all v1 preprocessors are replaced by v1.1 except those doesn't appear in v1.1.
NOTE: Please refrain from using the controlnet preprocessor alongside this installation, as it may lead to conflicts and prevent proper recognition.
Conflicted Nodes:
AIO_Preprocessor [ComfyUI-Inference-Core-Nodes], AnimalPosePreprocessor [ComfyUI-Inference-Core-Nodes], AnimeFace_SemSegPreprocessor [ComfyUI-Inference-Core-Nodes], AnimeLineArtPreprocessor [ComfyUI-Inference-Core-Nodes], BAE-NormalMapPreprocessor [ComfyUI-Inference-Core-Nodes], BinaryPreprocessor [ComfyUI-Inference-Core-Nodes], CannyEdgePreprocessor [ComfyUI-Inference-Core-Nodes], ColorPreprocessor [ComfyUI-Inference-Core-Nodes], DWPreprocessor [ComfyUI-Inference-Core-Nodes], DensePosePreprocessor [ComfyUI-Inference-Core-Nodes], DepthAnythingPreprocessor [ComfyUI-Inference-Core-Nodes], DiffusionEdge_Preprocessor [ComfyUI-Inference-Core-Nodes], FacialPartColoringFromPoseKps [ComfyUI-Inference-Core-Nodes], FakeScribblePreprocessor [ComfyUI-Inference-Core-Nodes], HEDPreprocessor [ComfyUI-Inference-Core-Nodes], HintImageEnchance [ComfyUI-Inference-Core-Nodes], ImageGenResolutionFromImage [ComfyUI-Inference-Core-Nodes], ImageGenResolutionFromLatent [ComfyUI-Inference-Core-Nodes], ImageIntensityDetector [ComfyUI-Inference-Core-Nodes], ImageLuminanceDetector [ComfyUI-Inference-Core-Nodes], InpaintPreprocessor [ComfyUI-Inference-Core-Nodes], LeReS-DepthMapPreprocessor [ComfyUI-Inference-Core-Nodes], LineArtPreprocessor [ComfyUI-Inference-Core-Nodes], LineartStandardPreprocessor [ComfyUI-Inference-Core-Nodes], M-LSDPreprocessor [ComfyUI-Inference-Core-Nodes], Manga2Anime_LineArt_Preprocessor [ComfyUI-Inference-Core-Nodes], MaskOptFlow [ComfyUI-Inference-Core-Nodes], MediaPipe-FaceMeshPreprocessor [ComfyUI-Inference-Core-Nodes], MeshGraphormer-DepthMapPreprocessor [ComfyUI-Inference-Core-Nodes], MiDaS-DepthMapPreprocessor [ComfyUI-Inference-Core-Nodes], MiDaS-NormalMapPreprocessor [ComfyUI-Inference-Core-Nodes], OneFormer-ADE20K-SemSegPreprocessor [ComfyUI-Inference-Core-Nodes], OneFormer-COCO-SemSegPreprocessor [ComfyUI-Inference-Core-Nodes], OpenposePreprocessor [ComfyUI-Inference-Core-Nodes], PiDiNetPreprocessor [ComfyUI-Inference-Core-Nodes], PixelPerfectResolution [ComfyUI-Inference-Core-Nodes], SAMPreprocessor [ComfyUI-Inference-Core-Nodes], SavePoseKpsAsJsonFile [ComfyUI-Inference-Core-Nodes], ScribblePreprocessor [ComfyUI-Inference-Core-Nodes], Scribble_XDoG_Preprocessor [ComfyUI-Inference-Core-Nodes], SemSegPreprocessor [ComfyUI-Inference-Core-Nodes], ShufflePreprocessor [ComfyUI-Inference-Core-Nodes], TEEDPreprocessor [ComfyUI-Inference-Core-Nodes], TilePreprocessor [ComfyUI-Inference-Core-Nodes], UniFormer-SemSegPreprocessor [ComfyUI-Inference-Core-Nodes], Unimatch_OptFlowPreprocessor [ComfyUI-Inference-Core-Nodes], Zoe-DepthMapPreprocessor [ComfyUI-Inference-Core-Nodes], Zoe_DepthAnythingPreprocessor [ComfyUI-Inference-Core-Nodes]

ComfyUI-Inference-Core-Nodes is not an independent nodes but a collection of nodes, including comfyui_controlnet_aux, meap158/ComfyUI-Prompt-Expansion, comfyanonymous/ComfyUI_experiments and huchenlei/ComfyUI-layerdiffuse.
So you can only choose one of the options, use the collection or the other 4 nodes.


Studio view code snippet and learn docs

TODO:

need to revisit template inheritance. Currently I've overwritten the openai-chat entry, but instead I need to have dirs for learndocs and viewcode (In any case, locally building the template is also now broken for me)

First attempt to create a template for chat completions in JS.
The aim is to merge the source materials from this Learn docs page, the AI Studio Chat playground "view code" for JS, and the existing ai cli chat sample.
Of note, I've added two new _TEMPLATE var definitions and one new var _IS_AUTHOR_COMMENT for adding comments to template itself.
Once I have these being generated correctly, I'll add tests to validate.
Then on to parameterizing and configuring source repos for embedding from this output.

Need to revisit template


[Source Request]

PLEASE READ THIS
I acknowledge that:

I have checked that the extension does not already exist:

In the list of existing extensions in the application
In the multi-source spreadsheet: https://tachiyomi.org/extensions
By searching the GitHub repository for the extension and verify it does not appear in the code base

I have searched the existing GitHub issues and this extension does NOT have an open request
I will fill out the title and the information in this template

Note that the issue will be automatically closed if you do not fill out the title or requested information.
DELETE THIS SECTION IF YOU HAVE READ AND ACKNOWLEDGED IT

Source website

Name: Anixlife
Link: https://anixlife.xyz/
Language(s): Indonesia

Other details
Streaming Anime & Donghua

MoonLGH bang bisa di buatkan yg ini gk, buat nonton donghua, soalnya blum ad nih yg donghua min

New domain https://anixverse.com/


Upgrade to Elements 3

(first off, sorry for the second attempt and chunky PR)
This updates to GOV.UK Elements 3.0.1 (changelog) and GOV.UK Frontend Toolkit 5.2.1 (changelog). There are a couple of minor accessibility improvements but the big changes are the removal of the JavaScript requirement for radio buttons and checkboxes, and the change in the class applied to form groups and controls that are invalid.
This change has necessitated reworking the markup for radios and checkboxes (along with the error class changes). Along the way I noticed a few places where common markup could be refactored so there are a couple of commits to address that. Hopefully the commits should be fairly readable in order, but if there are any questions then I’m happy to answer.

looking

generally LGTM!


Starknet contracts: ABIs and cross-contract interactions

Also now that I think about it using the IERC20 example might not be ideal as it's a bit too long and it's probably better if we can restrict the scope so that the reader can experiment with the examples themselves.
I'll leave it up to you, but this part will probably be reworked later with smarter examples. In the first Contract chapter we will very likely write a simple contract, and it would be cool if the user could write a second contract that makes calls to the contract he wrote previously, and finally test all of this in a third section.

Great PR! Thank you 💯
A few things to address:

Can you move the chapter09 before the Appendix in the summary
We should have an entire section dedicated to smart contracts, something like ch09-starknet-smart-contracts. The content you wrote would be a subsection of this section. Can you do this change as well?
I think we could merge these pages together to improve the structure of the book
Add Listing numbering (e.g. Listing 9-1: ERC20 Interface by Openzeppelin)

So here's the structure I ended up creating..

Starknet smart contracts  9
- ABIs and Contract Interactions  9.1
- ABIs and Interfaces 9.1.1
- Contract Dispatchers 9.1.2
- Library Dispatchers and system calls 9.1.3

The idea is sections under starknet contracts can go in as subsections, but can also be parent sections to other subsections too...does this make sense to do? Cause I feel like cramming all the subsections under let's say the ABIs and Contract Interactions into one page, might be too overwhelming for the reader

Cause I feel like cramming all the subsections under let's say the ABIs and Contract Interactions into one page, might be too overwhelming for the reader

Ok, let's keep it like this for now. We'll see if we end up with too many subsections, or if some can be merged, whether we group them or not.
However I still think at least ContractDispatcher and LibraryDispatcher could be grouped together

yes, we could have a dedicated section, that's practical oriented, walking the reader through all they've learnt from writing to deploying

This item belongs to payment request #31C0EE on OnlyDust:

from 0xBenaparte to Darlington02
1 item included
$188 for 3 hours of work

This item belongs to payment request #31C0EE on OnlyDust:

from 0xBenaparte to Darlington02
1 item included
$188 for 3 hours of work

Payment request #31C0EE has been processed and payment is complete. See you around on OnlyDust.


connection error between netopeer2-cli and netopeer2-server

Hi All
I am facing one issue while connecting netopeer2-cli with netopeer2-server
below is error

connect --host <IP_ADDRESS> --login root
nc ERROR: Unable to connect to <IP_ADDRESS> (Connection refused).
nc ERROR: SSH channel poll error ().
cmd_connect: Connecting to the <IP_ADDRESS>:830 as user "root" failed.

NOTE: 1.) netopeer2-server is running
2.) able to ping from both the sides

It may be a firewall problem, NETCONF SSH uses port 830, which need to be allowed.


IE11 issue

Hello,
When I try to view the live demo examples in Internet Explorer 11 I get nothing but a white screen and these errors in the console:

SCRIPT438: Object doesn't support property or method 'from'
and
SCRIPT5009: 'Dom7' is undefined

Is there a way to fix those?
Is there a version of Framework7 that is working(partly) on IE11?
Keep up the good work!

Sorry, but IE11 is not supported at all


tensorflow framework computation time

if framework=tensorflow is set, the computation_time set in the configuration file is not reflecting the true computation time
For example
with https://github.com/argonne-lcf/dlio_benchmark/blob/workloads/dlio_benchmark/configs/workload/resnet50.yaml
The actual tracing info is more than that was set.
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":110573,"ts":169653616,"dur":115326,"ph":"X","args":{"hostname":"x3101c0s7b0n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":110573,"ts":169778575,"dur":118006,"ph":"X","args":{"hostname":"x3101c0s7b0n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":110573,"ts":169910633,"dur":116634,"ph":"X","args":{"hostname":"x3101c0s7b0n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":110573,"ts":170239135,"dur":107555,"ph":"X","args":{"hostname":"x3101c0s7b0n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":110573,"ts":170356073,"dur":123732,"ph":"X","args":{"hostname":"x3101c0s7b0n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":110573,"ts":170487830,"dur":118801,"ph":"X","args":{"hostname":"x3101c0s7b0n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":110573,"ts":170613041,"dur":125267,"ph":"X","args":{"hostname":"x3101c0s7b0n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":110573,"ts":170744207,"dur":132214,"ph":"X","args":{"hostname":"x3101c0s7b0n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":110573,"ts":170883659,"dur":119470,"ph":"X","args":{"hostname":"x3101c0s7b0n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":110573,"ts":171017504,"dur":136645,"ph":"X","args":{"hostname":"x3101c0s7b0n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}

I replaced the tf.function with sleep
    def compute(self, epoch_number, step, computation_time):
        sleep(computation_time)
        #tf.function(self.model)(epoch_number, step, computation_time)

This PR addressed this issue
https://github.com/argonne-lcf/dlio_benchmark/pull/96

After this fix, the new event tracing info is
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":60648,"ts":26800954,"dur":100916,"ph":"X","args":{"hostname":"x3005c0s25b1n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":60648,"ts":27256902,"dur":101186,"ph":"X","args":{"hostname":"x3005c0s25b1n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":60648,"ts":27449306,"dur":100224,"ph":"X","args":{"hostname":"x3005c0s25b1n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":60648,"ts":27555048,"dur":106199,"ph":"X","args":{"hostname":"x3005c0s25b1n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":60648,"ts":27667991,"dur":100120,"ph":"X","args":{"hostname":"x3005c0s25b1n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":60648,"ts":27781787,"dur":102623,"ph":"X","args":{"hostname":"x3005c0s25b1n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":60648,"ts":27899123,"dur":99752,"ph":"X","args":{"hostname":"x3005c0s25b1n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":60648,"ts":28008068,"dur":105414,"ph":"X","args":{"hostname":"x3005c0s25b1n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}
{"name":"TFFramework.compute","cat":"ai_framework","pid":0,"tid":60648,"ts":28129563,"dur":99047,"ph":"X","args":{"hostname":"x3005c0s25b1n0","core_affinity": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63]}}

This is fixed by #96


Error in the documentation (wiki) of the core resolvers

In the wiki you describe how to extend core resolvers: https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Extensions#extending-core-resolver
In the example of the ModelResolver it says that in order to register one you have to use:
ModelConverters.getInstance().addConverter(new MyCustomResolver());
However, the class MyCustomResolver that you presented does not have a no-argument constructor. Therefore, the example as you provided it will not compile.
It would be great if that could be fixed.

fixed


Add data columns to join hash table payload

Analyzing profiles for TPC-H queries I found there are many cache misses when we access columns with scan index > 0 and this is the main performance issue for join queries. I tried to fight with those LLC misses by storing required columns in join hash tables as a payload. Basically this gives us row representation for joined tables which can significantly reduce number of LLC misses. In case join results small number or no matches at all this patch can harm performance because stored payload is not used and therefore increased hash table size and payload build time are not payed off.
The patch adds payload support only for JoinHashTable which is used by most of TPC-H queries. There are performance results (SF=30):

TPC-H
Ref (ms)
Patched(ms)
Gain

Q1
914
920
0.99347826

Q5
4029
2992
1.34659091

Q8
453
455
0.9956044

Q7
4552
3029
1.50280621

Q8
8124
5929
1.3702142

Q9
21860
15748
1.38811278

Q10
7342
5849
1.25525731

Q11
818
830
0.98554217

Q12
428
455
0.94065934

Q18
24406
25404
0.96071485

Q19
18278
9407
1.94302115

As expected we can get significant gain when stored payload is frequently used and some performance loss due to increased hash table size when payload is not used much.
Current implementation has some work to do:

row ids can be actually completely removed from the table is all required columns are stored as a payload. For one-to-one tables this requires some new way to detect empty hash entries. In case payload has not-null columns a NULL value can be used for that. This should decrease table size and hopefully reduce performance losses in bad scenarios
hash table cache can be improved to better re-use tables in case of partial table key match (e.g. we can re-use only a subset of columns from cached table)

Still I believe this patch is enough to start discussion of the approach and to collect some performance data. I put the feature under a flag to simplify performance measurements. For now it is disabled by default but I hope we can enable it by default eventually.

Hi @ienkovich , thank you for the patch. This is something we thought about quite a bit in the past, there is a case to be made for it and the benchmarks prove it. As a default policy, it has at least two drawbacks:

Decrease the cache hit rate. In the current world, we can reuse the same hash table with different payloads. If we fuse the payload into the hash table, that's not true anymore.
In certain cases (join on dictionary strings with different encodings) we're building the hash table on the host and use it on GPU. With this approach, we force the payload to be fetched on the host, which happens at a lower bandwidth than on GPU.

I don't think we'd be fine with taking any regression, so the right approach would be to collect runtime statistics about the workload and whether doing this would be advantageous. In other words, we'd have this as an additional policy which kicks in selectively, based on workload.
On the other hand, I see no obvious blocker to merging this behind the flag you've added, as long as all tests pass with the flag set. The smart policy which activates it can be follow-up work, there's no requirement as far as I'm concerned to bundle it with this work. We'll still need to review this carefully since it's a rather big change, but as long as changes are well isolated we should be able to take it. This has great potential, thanks!

I understand the initial version is not good enough to become a default strategy. For now it can be a tool to find cases when payload really helps. Hopefully we will come with some heuristics to enable it in most profitable cases.
I agree cache reusage becomes an issue. This first implementation simply checks payload is the same. This definitely should be relaxed to allow better reusage (probably even if required payload is partly cached). But anyway payload is basically an additional index to speed-up query and it has to come with additional maintenance cost.
I'm not sure fused payload can be profitable on GPU as much as on CPU and enabled it for CPU only.

Hi @ienkovich
I am having some problems building this with ENABLE_CUDA = on:
[ 47%] Linking CXX static library libthrift_handler.a
/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntime.cpp(124): error: identifier "fixed_width_int_decode_noinline" is undefined

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntimeGpu.cu(40): error: no suitable conversion function from "const JoinColumn" to "int32_t" exists

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntimeGpu.cu(41): error: no suitable user-defined conversion from "const JoinColumnTypeInfo" to "JoinColumn" exists

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntimeGpu.cu(42): error: no suitable constructor exists to convert from "long" to "JoinColumnTypeInfo"

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntimeGpu.cu(44): warning: integer conversion resulted in a change of sign

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntimeGpu.cu(45): error: argument of type "int" is incompatible with parameter of type "const void *"

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntimeGpu.cu(46): error: argument of type "int64_t" is incompatible with parameter of type "const void *"

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntimeGpu.cu(46): error: too few arguments in function call

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntimeGpu.cu(140): error: too few arguments in function call

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntimeGpu.cu(230): error: identifier "fill_row_ids_gpu" is undefined

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntimeGpu.cu(274): error: identifier "fill_row_ids_bucketized_gpu" is undefined

[ 47%] Built target thrift_handler
/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntime.cpp(838): error: identifier "payload" is undefined

/home/alexb/Projects/omniscidb/QueryEngine/HashJoinRuntime.cpp(838): error: identifier "payload_count" is undefined

12 errors detected in the compilation of "/tmp/tmpxft_000441b4_00000000-6_HashJoinRuntimeGpu.cpp1.ii".
make[2]: *** [QueryEngine/CMakeFiles/QueryEngine.dir/build.make:96: QueryEngine/HashJoinRuntimeGpu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 48%] Linking CXX static library libChunk.a
[ 48%] Built target Chunk
make[1]: *** [CMakeFiles/Makefile2:1379: QueryEngine/CMakeFiles/QueryEngine.dir/all] Error 2
make: *** [Makefile:163: all] Error 2

Were you able to build w/ CUDA enabled?

The patch was tested on not CUDA enabled machine. I'll fix the CUDA build.

If you have the cuda drivers but no GPUs you should still be able to build -- we need to make some modifications in our unit test framework to pass on a CUDA build with no GPUs available. If you don't want to pollute your env, nvidia-docker is a good option -- we have pre-built containers here: https://hub.docker.com/r/omnisci/core-build-centos7-cuda10.0

CUDA build should be fixed now but I found another issue, in some case a wrong payload is loaded into a hash table. I couldn't find the reason yet. Will look into it.

Fixed issue with wrong payload loaded.

This looks good, I've merged all the commits from our internal repo over and this repo is now fully up to date. Do you mind rebasing and fixing the conflicts? After that we can get it merged.

Thanks for review! I fixed conflicts.

@alexbaden Let's check that the IR is unchanged (if the flag is not set) before merging.

I compared generated IRs (using IR log channel) for TPC-H queries and found no difference.


BackEnd error handeling

Instantiating BackEnd should not open a connection, that functionality should be a callable method in order to handle errors. Method should probably throw errors in order to be handled by FrontEnd.

Actually, should be either/or. It should either throw an error, or it should return an error code.

Sounds good. Will do when I have time.
On Apr 9, 2015 9:48 PM, "Nolan Groves"<EMAIL_ADDRESS>wrote:

Actually, should be either/or. It should either throw an error, or it
should return an error code.
—
Reply to this email directly or view it on GitHub
https://github.com/rpbeltran/closed-kimono/issues/1#issuecomment-91410195
.

Fixed


Setup Tutorial?

I'm not great with scripting in general, so it would be great to know how to set this up!

Run the EXE application with commands (assuming you opened the console/powershell in the target folder use console!!). E.x: Unluau.CLI.exe (filename here mus be in the same folder) or unluau v2 Unluau.CLI.exe disassemble (filename here) (extra options)
to get options do Unluau.CLI.exe -h
if you need any more help ask chatgpt
it only works for bytecode version 5 tho


Pretty Please Fix Pretty Print

After the update, pretty print devolved into

where-as before it'd looked pretty

sorry for the big number, it's the image I could find

That's not my fault, that's because the format changed (although I'm surprised it changed like that). It used to be JSON, now it's apparently stringified JSON, which means that is the literal string that belongs in that field.


Feature Request - Add Labels for Displays

It would be very useful from a scripting perspective to be able to assign a label to a display and use that in DISPLAY_SEL.
For instance, 'left' or 'right' for displays is a lot easier to reason about than magical UUIDs, especially for commands involving sending spaces and windows to specific displays.

I use
yabai -m display --focus west
yabai -m display --focus south
yabai -m display --focus north
yabai -m display --focus east

And equally for moving windows
yabai -m window --display west
yabai -m window --display south
yabai -m window --display north
yabai -m window --display east

If you already have it in a script. I guess you could just assign it to something else
LEFT=<UUID-FOR-LEFT-SCREEN>
RIGHT=<UUID-FOR-RIGHT-SCREEN>

Thank you, @brorbw. That's a good set of suggestions.
Moving windows between displays is covered using the tools you've suggested, but sometimes I have need to move across multiple displays to get to the target.
I'm using screens by UUID currently for their various references, but this means I still have to store mappings between the UUID and 'human readable' display names - We're already attaching labels to spaces, so why not just add them as an optional field/reference in yabai?

This should cover moving across multiple displays
alt - h: yabai -m window --focus west  || (yabai -m display --focus west  && yabai -m window --focus east)
alt - j: yabai -m window --focus south || (yabai -m display --focus south && yabai -m window --focus north)
alt - k: yabai -m window --focus north || (yabai -m display --focus north && yabai -m window --focus south)
alt - l: yabai -m window --focus east  || (yabai -m display --focus east  && yabai -m window --focus west)

Equally for moving windows:
shift + cmd - h: yabai -m window --swap west  || (yabai -m window --display west && yabai -m display --focus west)
shift + cmd - j: yabai -m window --swap south || (yabai -m window --display south && yabai -m display --focus south)
shift + cmd - k: yabai -m window --swap north || (yabai -m window --display north  && yabai -m display --focus north)
shift + cmd - l: yabai -m window --swap east  || (yabai -m window --display east && yabai -m display --focus east)

On master it is possible to change how arrangement indices are interpreted (e.g. sort by x-coordinate/y-coordinate): https://github.com/koekeishiya/yabai/issues/550#issuecomment-1989676801
This should allow for a more intuitive usage of default indices (1 to display_count).
I am not against adding labels if that is still regarded as a useful addition.

Implemented labels for displays on master.

Fixed in v7.0.0.


Question about subscriber implementation

I was just browsing repository this is a minor thing, I came across subscriber implementation here platforms/x86_64/core/src/trace.rs
I am not sure if exit function implementation has intent to enter the span here -
https://github.com/tosc-rs/mnemos/blob/416b7d59fbc7fa889a774f54133786a584eb8732/platforms/x86_64/core/src/trace.rs#L90
Thanks

Nice catch, that's definitely a bug --- it looks like that would result in incorrectly-formatted traces being written to the serial port. We should change that to serial.exit(span).
I think the main reason this wasn't caught earlier is because the x86_64 platform implementation for MnemOS doesn't actually have a working serial driver yet, so we weren't actually writing anything to the serial port in the subscriber, just to the framebuffer. So, nobody noticed wrong traces being written to the serial port because we don't actually do that yet.
If you want to open a PR to change this to call serial.exit(span) instead, I'd be happy to merge it. Otherwise, I'll go ahead and fix this. Thanks for the bug report!

I will open PR today, no worries.
Thank you


Dramatic difference in perf between 1080ti and VEGA FE

Hey there,
I'm trialing some code to benchmark my VEGA vs a colleagues 1080ti.
I've noticed some very peculiar differences in time to fit per epoch, I'm guessing I'm messing up in some way.
For one epoch on AMD 2990WX ~400 seconds
For one epoch on 1080ti  < 100s
For one epoch on VEGA FE > 40 minutes

Hi @Dekken , could you help fill the issue template so we can understand the issue better?
https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/blob/develop-upstream/.github/ISSUE_TEMPLATE/00-bug-performance-issue.md

Have I written custom code: yes
OS Platform and Distribution: Linux Ubuntu 18
TensorFlow installed from source
TensorFlow version: ROCm dev
Python version: 3.7
Bazel version: .21
GCC/Compiler version 8.2
ROCm/MIOpen version: latest from xenial repo
GPU model and memory: VEGA FE 16GB
== cat /etc/issue ===============================================
Linux ws156 4.18.0-15-generic #16-Ubuntu SMP Thu Feb 7 10:56:39 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
VERSION="18.10 (Cosmic Cuttlefish)"
VERSION_ID="18.10"
VERSION_CODENAME=cosmic

== are we in docker =============================================
No

== compiler =====================================================
c++ (Ubuntu 8.2.0-7ubuntu1) 8.2.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

== uname -a =====================================================
Linux ws156 4.18.0-15-generic #16-Ubuntu SMP Thu Feb 7 10:56:39 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

== check pips ===================================================
numpy                    1.16.1
numpydoc                 0.8.0
protobuf                 3.6.1
tensorflow               1.12.0
tensorflow-estimator     1.13.0rc0

== check for virtualenv =========================================
False

== tensorflow import ============================================
Limited tf.compat.v2.summary API due to missing TensorBoard installation
tf.VERSION = 1.12.0
tf.GIT_VERSION = merge-190218-2-gd5532be4
tf.COMPILER_VERSION = merge-190218-2-gd5532be4
Sanity check: array([1], dtype=int32)

== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================
./sys.sh: line 105: nvidia-smi: command not found

== cuda libs  ===================================================

Thanks @Dekken , first I'd suggest is to try our docker image, the goal is to isolate the issue to either user-bit configurations or kernel drivers/firmware.
Can you refer to the following comment and get us the perf results of resnet50?
https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/issues/173#issuecomment-465871004

I'd rather not install docker if it's ok
TF_ROCM_FUSION_ENABLE=1 python3 tf_cnn_benchmarks.py --num_gpus=1 --batch_size=128 --model=resnet50

Done warm up
Step    Img/sec total_loss
1       images/sec: 207.5 +/- 0.0 (jitter = 0.0)        7.972
10      images/sec: 207.0 +/- 0.2 (jitter = 1.1)        7.856
20      images/sec: 206.8 +/- 0.2 (jitter = 0.5)        7.914
30      images/sec: 206.8 +/- 0.2 (jitter = 0.5)        7.734
40      images/sec: 206.8 +/- 0.1 (jitter = 0.6)        7.969
50      images/sec: 206.7 +/- 0.1 (jitter = 0.5)        8.025
60      images/sec: 206.7 +/- 0.1 (jitter = 0.6)        7.896
70      images/sec: 206.8 +/- 0.1 (jitter = 0.6)        7.991
80      images/sec: 206.8 +/- 0.1 (jitter = 0.6)        7.816
90      images/sec: 206.8 +/- 0.1 (jitter = 0.5)        7.799
100     images/sec: 206.7 +/- 0.1 (jitter = 0.6)        7.821
----------------------------------------------------------------
total images/sec: 206.71
----------------------------------------------------------------

TF_ROCM_FUSION_ENABLE=1 python3 tf_cnn_benchmarks.py --num_gpus=1 --batch_size=128 --model=resnet50 --use_fp16
Step    Img/sec total_loss
1       images/sec: 198.6 +/- 0.0 (jitter = 0.0)        7.878
10      images/sec: 198.8 +/- 0.3 (jitter = 1.1)        7.958
20      images/sec: 199.3 +/- 0.2 (jitter = 0.6)        7.949
30      images/sec: 199.3 +/- 0.2 (jitter = 0.6)        7.948
40      images/sec: 199.4 +/- 0.1 (jitter = 0.7)        7.967
50      images/sec: 199.4 +/- 0.1 (jitter = 0.7)        7.706
60      images/sec: 199.4 +/- 0.1 (jitter = 0.7)        7.923
70      images/sec: 199.4 +/- 0.1 (jitter = 0.6)        7.849
80      images/sec: 199.4 +/- 0.1 (jitter = 0.5)        7.969
90      images/sec: 199.3 +/- 0.1 (jitter = 0.5)        7.810
100     images/sec: 199.3 +/- 0.1 (jitter = 0.6)        7.785
----------------------------------------------------------------
total images/sec: 199.25
----------------------------------------------------------------

Hmm, the perf numbers look normal for Vega FE workstation card.
Can we have scripts/steps to repro your perf issue?

my test case is about 30GB, I'll see if I can minimize it, it's not mine

for sake of completeness, there's a 6GB archive available here https://hephaistos.lpp.polytechnique.fr/data/jeandet/dekken.tgz
extract and run test.py
CPU only runs into segfaults mind yuou

Thanks @Dekken , let me try it out.

mine with i76950x
CUDA_VISIBLE_DEVICES="" python3 test.py
Epoch 1/10
/home/philix/.local/lib/python3.7/site-packages/keras/engine/training_utils.py:481: UserWarning: Found both `sample_weight` and `class_weight`: `class_weight` argument will be ignored.
  warnings.warn('Found both `sample_weight` and `class_weight`: '
2019-02-21 22:28:37.008764: E tensorflow/stream_executor/rocm/rocm_driver.cc:1020] could not retrieve ROCM device count: HIP_ERROR_NoDevice
10229/10229 [==============================] - 205s 20ms/step - loss: 0.0491 - val_loss: 0.0161
Epoch 2/10
10229/10229 [==============================] - 204s 20ms/step - loss: 0.0383 - val_loss: 0.0141
Epoch 3/10
10229/10229 [==============================] - 204s 20ms/step - loss: 0.0348 - val_loss: 0.0137
Epoch 4/10
10229/10229 [==============================] - 203s 20ms/step - loss: 0.0330 - val_loss: 0.0123
Epoch 5/10
10229/10229 [==============================] - 203s 20ms/step - loss: 0.0317 - val_loss: 0.0118
Epoch 6/10
 7636/10229 [=====================>........] - ETA: 49s - loss: 0.0307Segmentation fault (core dumped)

@Dekken , can you post the log without CUDA_VISIBLE_DEVICES=""?
The environment variable works similarly to HIP_VISIBLE_DEVICES, with that being set, tensorflow-rocm won't be able to see or use any GPUs.

the normal log is what takes 40+ minutes per epoch - so I'd rather not do that completely
2019-02-21 23:01:58.296943: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1673] Adding visible gpu devices: 0, 1
2019-02-21 23:01:58.296970: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-02-21 23:01:58.296978: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1090]      0 1
2019-02-21 23:01:58.296986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1103] 0:   N N
2019-02-21 23:01:58.296993: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1103] 1:   N N
2019-02-21 23:01:58.297052: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1220] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 15306 MB memory) -> physical GPU (device: 0, name: Vega 10 XTX [Radeon Vega Frontier Edition], pci bus id: 0000:03:00.0)
2019-02-21 23:01:58.314591: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1220] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 15306 MB memory) -> physical GPU (device: 1, name: Vega 10 XTX [Radeon Vega Frontier Edition], pci bus id: 0000:0c:00.0)
2019-02-21 23:02:03.002830: I tensorflow/core/kernels/conv_grad_input_ops.cc:997] running auto-tune for Backward-Data
2019-02-21 23:02:03.078605: I tensorflow/core/kernels/conv_grad_filter_ops.cc:886] running auto-tune for Backward-Filter
2019-02-21 23:02:03.164291: I tensorflow/core/kernels/conv_grad_input_ops.cc:997] running auto-tune for Backward-Data
2019-02-21 23:02:03.262944: I tensorflow/core/kernels/conv_grad_filter_ops.cc:886] running auto-tune for Backward-Filter
2019-02-21 23:02:03.310904: I tensorflow/core/kernels/conv_grad_filter_ops.cc:886] running auto-tune for Backward-Filter
  222/10229 [..............................] - ETA: 40:30 - loss: 0.0836

Okay, I can repro your issue now.
Looks like the GPUs were under very light loads during training. Next step would be to profile the workloads on ROCm and the other targets, and look into the device placement of each ops, please stay tuned.

Did you confirm CPU segfault by any chance?

Hi @Dekken , the CPU segfault was not observed on my local system.
While we look into the issue, can you share a bit background of the case? e.g. is it based on a public project? Or are there any similar public models we can try as well?

It's not my project so I'm asking to get others in.
I'm going to see if I can get a colleague in.

pinging @gautiernguyen @nicolasaunai

I think my segfault was to OOM

My colleagues are currently on vacation but this is a part of a PhD in the plasma physics department around "Earth's magnetopause detection from in situ satelites measurements with ML"
not that I know what this means

Made a bit of progress week.  Turns out we were using some non-optimal reduction code due to some stale ROCm-related ifdefs.  We have some local changes that show significant improvements.
Here's where the (initial) bottleneck occurs:
https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/blob/develop-upstream/tensorflow/core/kernels/reduction_gpu_kernels.cu.h#L713-L736
Btw - thanks for reporting this issue.

@parallelo can’t wait for your PR J

@whchung
Currently wrapping up some mods.  It is running now, and it appears to be converging (as best I can tell).
There's always room for more improvement, but the initial reduction kernel bottleneck appears to be fixed  :-)

with #344 merged, should we be good?
if I compile "develop-upstream" myself I should be able to confirm?

Hi @Dekken - Worth another try now.  Please check and report back here.

there's certainly a differnce
we're at 300 seconds an epoch

should we in theory be on par (or better) than a 1080ti with a VEGA FE?

Hi @Dekken , the TF-ROCm 1.14 release includes the PR #344. Can you check it there and let us know if it's okay to close this issue?

is there somewhere to continue to assess the performance vs a 1080ti?
it's a bit disappointing still

is there somewhere to continue to assess the performance vs a 1080ti?

@Dekken , one path is to profile your workload using RPT:
https://scchan.github.io/hcc/md__home_scchan_code_hcc_doc_markdown_hcc_profile.html
And then cross-compare the nvprof results from 1080ti. With that, we might be able to figure out if there's any other avenue or low hanging fruit to improve the performance.


No Supported Code Available

I have reproduced this issue by simply checking out https://github.com/Xcaciv/dvta opening DVTA.sln and trying to scan it. There are blatant issues with the code like SQL Injection, but it does not see any files as scannable. The Snyk window shows the following:

Log file attached
snyk-extension.test.log

Hi @Xcaciv,
Thank you!

@Xcaciv I just want to double check that version of Snyk extension you use?

I am using 1.1.20. I just updated to get the changes for SCA yesterday.

Hi @Xcaciv 👋
We released new version (1.1.22) with the fix for this issue. Can you please install the new version and try it?
Best regards,
Alex

Hi, since GitHub issues are not officially monitored by Snyk support, we will be disabling "Issues" tab on this repository very soon.
Please raise the current issue directly by contacting Snyk Support on the web or emailing us at<EMAIL_ADDRESS>

This does find files now. However, there are no results, and this is consistent with the CLI and SaaS for the sited repo.


debian-6 is EOL next month

t1.micros are also beyond terrible for performance.

@jkeiser

We should remember to pull it from https://github.com/chef/chef-rfc/blob/master/rfc021-platform-support-policy.md in a month


A simple enhancement, in dataset creation time

Hi Team,
recently, i was using phi3 model to train model on binary classification datatset.
Despite training it for good amount of time, when i try to use it, it give me output but fails to stop after giving class name.
Problem : in current code there is no concept of adding eos_token . Hence after giving otuput it start giving prompt back.
so i made slight change in dataset creation function and after that, i trained the model again and it worked like charm.
i was not getting one word output as i want. there is no more un neccessary extra answer.
`class ChatDataset(Dataset):
def __init__(self, path: Path, tokenizer: PreTrainedTokenizer):
    super().__init__(path)
    self._tokenizer = tokenizer

def __getitem__(self, idx: int):
    messages = self._data[idx]["messages"]
    text = self._tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )
    text = text+self._tokenizer.eos_token   ## this is new addition of code.
    # print(text)
    return text`

requesting github repo owner, to kindly look into this
and update the code..thanks

Thanks. It looks like most tokenizers do not default append the eos token id. So we can append it manually when fine-tuning. I'm just wondering if we should make it an option or not.. most likely the typical case will want the eos token appended to each example.


Migrate to 07/25 binaries

[ ] LoggerAPI (not needed)
[ ] SwiftyJSON
[ ] KituraSys
[ ] KituraNet
[ ] Kitura
[ ] HeliumLogger
[ ] KituraSample
[ ] CouchDB
[ ] Redis
[ ] Kitura-TemplateEngine
[ ] Kitura-Session
[ ] KituraCredentials
[ ] KituraCredentialsHTTP
[ ] KituraCredentialsGoogle
[ ] KituraCredentialsFacebook
[ ] Kitura-Credentials-Sample
[ ] Kitura-Session-Redis
[ ] GRMustache
[ ] Kitura-MustacheTemplateEngine
[ ] Kitura-Cache
[ ] Kitura-Compression

Actually, there is a 07-28 snapshot already...

Sorry if I sound obtuse, but what is the rationale behind not skipping ahead the the 28 instead of the 25 binaries? I don't recall any major changes to my own porting efforts, but I do not deal with hugely complex issues.

There isn't a working libdispatch for Linux available for the 07-28 toolchain, unfortunately.

All "Kitura" repositories have now been migrated to Swift 07/25.
The README.md files that mention version of Swift 3, all still say 06/20.


Immutable.Map Flowtype

Is there a way to type a Immutable.Map yet with the latest version of flow-type and is this on the roadmap?
If its possible, is there an example I can look at?

I think you can get the type by doing.
import type { Map } from 'immutable';

@athomann do you know how to use that type then?

I think You can do something like Map<string, string>.  Related issue: https://stackoverflow.com/questions/37033339/how-to-describe-immutable-js-map-shape-with-flow

Maps can be flow typed with Map<KeyType, ValueType>
Thanks for answering @athomann !


Implement endpoint for retrieving job log

Superseded by https://github.com/w3c/aria-at-app/pull/772


Implement Min and Max pushdown optimization

Do everything except for the storage bits, which is a separate issue.
NOTE: Currently each planner rule uses its own feature flag for min/max. It would be simpler to consolidate those flags to be a single flag for min and a single flag for max. Meaning that each planner rule would consult the same flag for min and max respectively.
DOD:

End to end _test.flux tests that demonstrate correctness
launcher test for OSS
Pipeline for idpe
Add test cases to verify that rule works right
Turn on feature flag in tools, and then production clusters

Deployment steps are:

Turn on the feature flag for your user in tools. Test manually, show that it's correct, and if relevant, faster
Once your satisfied, enable for the whole tools cluster. Let bake for an hour at least (look at queryd health dashboard)
Deploy to production clusters, start with us-west-2, 25% of traffic. Bake for an hour
Turn it on for all of us-west-2, let bake 1 hour
Turn on everywhere

All of the test cases for this are complete so we are going through the process of toggling the feature flags.

This is on in tools and we're going to let it bake for a week before turning it on in production.

This was intended to include group |> min/max so I'll write the tests for that and then close this.

There's more extensive work to do with group so we will cover it as part of a separate issue.


Sous Chefs Adoption

Description
Change appropriate files to reflect Sous Chefs adoption
Issues Resolved
None
Check List

[X] All tests pass. See TESTING.md for details.
[X] New functionality includes testing.
[X] New functionality has been documented in the README if applicable.

Released as: 5.1.0


Metrics stop collecting after some time

Metrics are collecting fine then it will just stop after some time.  The error shows:
"Unexpected error raised. Stopping the timer. title=:metric_scraper error_class=RestClient::Unauthorized error="401 Unauthorized"
The only way for it to collect metrics again is to restart the metrics pod.
This issue is similar to
https://github.com/splunk/splunk-connect-for-kubernetes/issues/204
but instead of metrics-aggr pod,  this is for metrics pod.     I saw that there is a code fix for metrics-aggr pod, is that fix applicable to metrics pod as well?
Thanks

what version of the pod are you using?

@matthewmodestino
Hi Matt, metrics pod version is 1.1.0.    Kube is 1.12.4.  Thanks

Are your kubelet certs self-signed?
This usually happens when that is the case.
If the certs are self-signed, just use insecure mode for now.

also please update to the latest metrics pods, they are now version 1.1.2.
Any chance this is GKE?

@chaitanyaphalak.   Yes, it is self-signed.  but cannot make it insecure mode unfortunately.   The fix discussed in #204 for metricsaggr, its not applicable here?  The symptoms are exactly the same.
@matthewmodestino   not GKE.  its an on-prem kube.  I will try 1.12.

@beth2016 that fix is for a different issue, unfortunately, with our current implementation, we do not support self-signed certificates.

@beth2016 self-signed certs, by definition are insecure. So setting insecureSSL to true just means we dont validate them, but it is still sending ssl...
Anyways, 401 unauth i believe to be unrelated to that...

sure, but that would just cover the root cause. Lets see if the updated pods gave the same issue..

are your certs being refreshed after a few hours too?

@matthewmodestino Hi Matt, can you pls guide me on how to set the liveness probe for that error message?  The network disconnect can be due to our envionments.

This looks like a problem in your environment.

@beth2016 can you share the settings/configs you are using?
Meanwhile, I would recommend you to try these settings,
kubernetes:
insecureSSL: true
useRestClientSSL: true
kubeletPort: 10250

Once a node is rebooted, splunk metrics will disconnect because it failed to connect to that node (rightfully so).  But after the node is powered up,  it still doesn't collect metrics.   The only way to get it to start collecting again is to restart the metrics pod.

looks more like an issue with k8s than SCK - https://github.com/kubernetes/kubernetes/issues/57739 - https://github.com/kubernetes/kubernetes/issues/31123

Closing due to inactivity


read_zipfile_from_stream takes reference to R: Read, instead of R: Read

I'm currently encountering an issue in design of a program that uses the zip crate centered around the function zip::read::read_zipfile_from_stream. I'm using it to create a zipfile struct, however, because read_zipfile_from_stream takes a reference to something implementing the read trait, instead of something taking the read trait itself, I'm finding it impossible to use this function in this instance. That's because I actually pass the ZipFile struct up out of the body where the Vec, and it's corresponding slice, are created that initialize the ZipFile, meaning that I end up invalidating the reference, because the reference refers to the Vec, however, the vec is dropped at the end of the function, whereas the ZipFile is supposed to live beyond that. This issue is intractable because I cannot move where I read the bytes from the stream a level up, and I have designed my own interfaces to take not a reference to something implementing Read, but to borrow something implementing Read. For the reasons above, I need a function exactly like read_zipfile_from_stream, but that does not take a reference. Is this possible to add to the zip crate? Its presence would make possible a much wider range of use cases for read_zipfile_from_stream than are currently possible.
In practice, here's how this occurs:
let buf: Vec<u8> = Vec::new();
// ...Omitted for brevity, but read into buf all of the bytes from some file ...
// Now internally use get_zipfile_from_stream to create a ZipFile from a reference to a buffer
let zipfile: ZipFile = getzipfilefrombufref(&buf[..]);
// whoops, we're about to return something that contains a reference to a buf that is about to go out of // scope
return zipfile;

In practice, it looks like this may be difficult because you've implemented everything in terms of taking references: for instance, your interfaces in podio that are relied upon take mutable references to read. I will think about how this might be overcome.

It does not really make sense to take ownership of the reader for most use-cases.
When doing that you can call the iterator function only once, for a single zip file.
Why not just return the Vec and parse the zipfile at a later time? Or use the normal zip parsing methods?

An owned generic parameter wouldn't need to take ownership of the reader - &mut impl Read: Read. I see no reason not to make this fix - The code changes should be fairly mechanical, but will be easier once we've cleaned up the open PRs.
One little thing is that this will have to wait for the next (pre-)release since it would technically be a breaking change for some edge cases.

I also found this API particularly troublesome for composition:
Imagine you have a Reader over a File and you want to wrap it with some sort of ZipDecoder which gives you a Read interface for decompressed file (assuming single-file archive) and then you do some post-processing e.g. Transcode which can also be abstracted with the Read trait...
Having &mut API that returns ZipFile<'a> with lifetime bound to the reader makes composition impossible. You can't have something like:
struct ZipDecoder<R: Read> {
  reader: R,
  zip_file: ZipFile<?> // references reader somehow???
}

because as far as I understand one member holding a &mut to another member of the same struct is impossible in rust.

I also found this API particularly troublesome for composition:

Yep, I also had this problem. I wanted to create a reader that concatenates multiple files inside a zip file to one contiguous reader. I ended up using unsafe and transmute to achieve what I wanted. The API is certainly somewhat hostile, but I don't know how you could do it better.

@Plecra's argument makes sense, this indeed makes more sense.

We have a slight issue in the implementation: A File that borrows from an Archive is different to one that owns its reader.
I doubt it'll be possible to make them the same type, which means we aren't going to be able to have a Vec<ZipFile> that could store files from both a file and stream. Will this be a problem for anyone?


Support sparse matrix with bool

Description
https://github.com/cupy/cupy/blob/eb4a7b4ca9b1caf139abac00845028e6419d51dc/cupyx/scipy/sparse/_coo.py#L141-L144
Additional Information
No response

Compressed matrices (csr_matrix, etc.) support bool.
https://github.com/cupy/cupy/blob/42bf511300cf0f37cc4b312e2b9518486d0e99da/cupyx/scipy/sparse/_compressed.py#L277

@kmaehashi Bool csr_matrix has not been tested, and many tests fail when bool is added to the testing combination.
https://github.com/cupy/cupy/blob/d21f772e5e50198e62275bf70683b2a947191c26/tests/cupyx_tests/scipy_tests/sparse_tests/test_csr.py#L136

I see, makes sense.

@asi1024 FYI - I've rewrote the issue body to use this issue as a tracker.


chore: merge release/1.2.0 to master

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: dda2bc0122d6e5d0e0599805d4d3bbe560b0703d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


The Semantic Camera Values need a paragraph on how to set them on objects in the Editor.

Add screen shot and some text on how to modify/edit/add the semantic camera value, i.e., set the custom depth stencil value.
https://monodrive.readthedocs.io/en/latest/monoDrive_home/Camera/

@stephendb, I will need more information to add this to the documentation because I am not familiar with how to do this. Would you be able to write up this description or can you recommend someone who can help with this specific task?

Add a how to add semantic labels to my doc
https://docs.google.com/document/d/1iljdCm0Qnd2K79vM236mpoWLb_wSeqpb6YmC9m_8JIs/edit?usp=sharing

Add a how to add semantic labels to my doc
https://docs.google.com/document/d/1iljdCm0Qnd2K79vM236mpoWLb_wSeqpb6YmC9m_8JIs/edit?usp=sharing

Brett, can we add to the this document why we want to add this values? i.e. for the semantic camera to paint the actor correctly, etc?
Also, is the link you have referenced the correct location to the documentation?

Brett, can we add to the this document why we want to add this values? i.e. for the semantic camera to paint the actor correctly, etc?
Also, is the link you have referenced the correct location to the documentation?

Sure I can add a 'why', also the last time I checked that is the spot that has the depth values so the link should be good. But I'll check and see if Katie changed it.

Sure I can add a 'why', also the last time I checked that is the spot that has the depth values so the link should be good. But I'll check and see if Katie changed it.

The Semantic labeling has been addressed in this document: https://docs.google.com/spreadsheets/d/148UZ8KIqxBZljg8LNLUnIMIKXu5U8upATmI-fUGp2ms/edit?usp=sharing


Steam-Prefill Failed on Some Games; /depot chunk errors

Describe the bug
When running Steam-Prefill, some games either have failed downloads, or fail entirely. For example, Age of Empires IV: Anniversary Edition has 43864 failed downloads, and Among Us has 715 failed downloads. Baldur's Gate 3 progress bars gets to 100%, then goes back to 0 and says Retrying 1, then Retrying 2 once it gets back to 100% again.  The attached app log had to be downsized due to size limits.
Please note, I have a thread in EpicPrefill where I mention a Fortnite problem. This Steam issue is happening after I wiped the cache drive (6TB HDD) and formatted it with brtfs.
Additional Details

unRAID Server using Lancache, Lancache-DNS, and Lancach-Prefill in Docker
ZurkeyDurkey
Region : US West Coast

Screenshots
If applicable, add screenshots to help explain your problem.

app (edit).txt

Since the today update to the new steamprefill 2.4.0 I got the same error:
Feb 12 18:06:41 HMLGC01 SteamPrefill[5259]: [6:06:41 PM] Starting GROUND BRANCH Feb 12 18:06:41 HMLGC01 SteamPrefill[5259]:  Getting available CDN Servers... Feb 12 18:06:41 HMLGC01 SteamPrefill[5259]:  Getting available CDN Servers...  Retrying 1 Feb 12 18:06:42 HMLGC01 SteamPrefill[5259]:  Getting available CDN Servers...  Retrying 2 Feb 12 18:06:43 HMLGC01 SteamPrefill[5259]:  Getting available CDN Servers...  Retrying 3 Feb 12 18:06:44 HMLGC01 SteamPrefill[5259]: Fetching depot manifests... Feb 12 18:06:55 HMLGC01 SteamPrefill[5259]: Downloading..: 0% Feb 12 18:07:25 HMLGC01 SteamPrefill[5259]: Downloading..: 2% Feb 12 18:07:55 HMLGC01 SteamPrefill[5259]: Downloading..: 4% Feb 12 18:08:25 HMLGC01 SteamPrefill[5259]: Downloading..: 6% Feb 12 18:08:55 HMLGC01 SteamPrefill[5259]: Downloading..: 8% Feb 12 18:09:25 HMLGC01 SteamPrefill[5259]: Downloading..: 10% Feb 12 18:09:55 HMLGC01 SteamPrefill[5259]: Downloading..: 12% Feb 12 18:10:25 HMLGC01 SteamPrefill[5259]: Downloading..: 14% Feb 12 18:10:55 HMLGC01 SteamPrefill[5259]: Downloading..: 16% Feb 12 18:11:18 HMLGC01 SteamPrefill[5259]: [6:11:18 PM] Request /depot/16901/chunk/d7978bd78f72acfcf696615eb69f9afe6febe795 Feb 12 18:11:18 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:11:25 HMLGC01 SteamPrefill[5259]: Downloading..: 17% Feb 12 18:11:55 HMLGC01 SteamPrefill[5259]: Downloading..: 19% Feb 12 18:12:14 HMLGC01 SteamPrefill[5259]: [6:12:14 PM] Request /depot/16901/chunk/fba3918b065e4e9894d86e7fd8958e20875a2e08 Feb 12 18:12:14 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:12:25 HMLGC01 SteamPrefill[5259]: Downloading..: 21% Feb 12 18:12:55 HMLGC01 SteamPrefill[5259]: Downloading..: 23% Feb 12 18:13:17 HMLGC01 SteamPrefill[5259]: Downloading..: 25% Feb 12 18:13:47 HMLGC01 SteamPrefill[5259]: Downloading..: 26% Feb 12 18:14:17 HMLGC01 SteamPrefill[5259]: Downloading..: 28% Feb 12 18:14:43 HMLGC01 SteamPrefill[5259]: [6:14:43 PM] Request /depot/16901/chunk/003fea2229e011dd1023923356d5d6a457153a94 Feb 12 18:14:43 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:14:47 HMLGC01 SteamPrefill[5259]: Downloading..: 30% Feb 12 18:15:17 HMLGC01 SteamPrefill[5259]: Downloading..: 32% Feb 12 18:15:47 HMLGC01 SteamPrefill[5259]: Downloading..: 34% Feb 12 18:15:59 HMLGC01 SteamPrefill[5259]: [6:15:59 PM] Request /depot/16901/chunk/4f6094f7333633d5b98baf48d0f31d5559480ca9 Feb 12 18:15:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:16:01 HMLGC01 SteamPrefill[5259]: [6:16:01 PM] Request /depot/16901/chunk/9c5477b54a551ef127c1ecbd645884f00fd4937a Feb 12 18:16:01 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:16:03 HMLGC01 SteamPrefill[5259]: [6:16:03 PM] Request /depot/16901/chunk/5e46197d61e50924fd7a08e55aeb51248667f712 Feb 12 18:16:03 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:16:17 HMLGC01 SteamPrefill[5259]: Downloading..: 36% Feb 12 18:16:36 HMLGC01 SteamPrefill[5259]: [6:16:36 PM] Request /depot/16901/chunk/826e53622a9c6cf9fcc48a414fe66b3179671562 Feb 12 18:16:36 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:16:47 HMLGC01 SteamPrefill[5259]: Downloading..: 38% Feb 12 18:17:17 HMLGC01 SteamPrefill[5259]: Downloading..: 40% Feb 12 18:17:47 HMLGC01 SteamPrefill[5259]: Downloading..: 42% Feb 12 18:18:04 HMLGC01 SteamPrefill[5259]: [6:18:04 PM] Request /depot/16901/chunk/9ec217b62909b9fd601af19c3c829e336f4435ed Feb 12 18:18:04 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:18:17 HMLGC01 SteamPrefill[5259]: Downloading..: 44% Feb 12 18:18:17 HMLGC01 SteamPrefill[5259]: [6:18:17 PM] Request /depot/16901/chunk/064e63c48969547d2df8bdbe4bda8952d9f88ce2 Feb 12 18:18:17 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:18:47 HMLGC01 SteamPrefill[5259]: Downloading..: 45% Feb 12 18:18:52 HMLGC01 SteamPrefill[5259]: [6:18:52 PM] Request /depot/16901/chunk/af6b9ef13521491d5376096afe76150776b8efb3 Feb 12 18:18:52 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:19:17 HMLGC01 SteamPrefill[5259]: Downloading..: 47% Feb 12 18:19:47 HMLGC01 SteamPrefill[5259]: Downloading..: 49% Feb 12 18:19:48 HMLGC01 SteamPrefill[5259]: [6:19:48 PM] Request /depot/16901/chunk/a30bc66eea464507b2287acabef6fd7cb66c91a7 Feb 12 18:19:48 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:19:51 HMLGC01 SteamPrefill[5259]: Downloading..: 50% Feb 12 18:19:56 HMLGC01 SteamPrefill[5259]: [6:19:56 PM] Request /depot/16901/chunk/1b45557793d88f652ff6b058ed33b8ff76676012 Feb 12 18:19:56 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:20:07 HMLGC01 SteamPrefill[5259]: [6:20:07 PM] Request /depot/16901/chunk/4201ee786e198703a80d1a36437369365dc087d7 Feb 12 18:20:07 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:20:21 HMLGC01 SteamPrefill[5259]: Downloading..: 51% Feb 12 18:20:51 HMLGC01 SteamPrefill[5259]: Downloading..: 53% Feb 12 18:21:08 HMLGC01 SteamPrefill[5259]: [6:21:08 PM] Request /depot/16901/chunk/5395f3b809ccb556f7de2f91d85c0a60732fcbc9 Feb 12 18:21:08 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:21:21 HMLGC01 SteamPrefill[5259]: Downloading..: 55% Feb 12 18:21:30 HMLGC01 SteamPrefill[5259]: [6:21:30 PM] Request /depot/16901/chunk/53ea4d4f2262af9eda2536e0ad9b3f83a4b3cffb Feb 12 18:21:30 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:21:31 HMLGC01 SteamPrefill[5259]: [6:21:31 PM] Request /depot/16901/chunk/f78c6971193cc41ca776f26407d5d10f5ce2917f Feb 12 18:21:31 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:21:51 HMLGC01 SteamPrefill[5259]: Downloading..: 57% Feb 12 18:22:21 HMLGC01 SteamPrefill[5259]: Downloading..: 59% Feb 12 18:22:41 HMLGC01 SteamPrefill[5259]: [6:22:41 PM] Request /depot/16901/chunk/8eba3dec1d801c0164c3cbc7a63f36f053d95cb4 Feb 12 18:22:41 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:22:51 HMLGC01 SteamPrefill[5259]: Downloading..: 61% Feb 12 18:23:18 HMLGC01 SteamPrefill[5259]: [6:23:18 PM] Request /depot/16901/chunk/4a4bb54e7f2b6541c068a4049a6de2c3594228fb Feb 12 18:23:18 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:21 HMLGC01 SteamPrefill[5259]: Downloading..: 63% Feb 12 18:23:51 HMLGC01 SteamPrefill[5259]: Downloading..: 65% Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/9cba303bf5f2424449b018aa3896b9effe84f430 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/f5c2be8f10ed308a9f8b780024fe14b34c68212b Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/b3076a42bf4ea38de52b2ab407054500817f1136 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/2ce6e2702fbe0ed5bff7cbb36e09dbaacd48dbcd Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/849d535928de68f0b798f3e358357580fb754f46 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/933949e952dbdf86e560382ff83dffcff4a59479 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/da2dd15bf5281d1f74d9c82d4871002ed8b0c432 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/757f55c3ea1f4597154b948a4afcd04913c3fdc7 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/7ec59ffc4f1111f35d0ee161300a7ccb3b343a3e Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/b707778bcf81679266057195e31a11d6f884b857 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/8685530b3639b954983e55d0efdf855ca04bcef1 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/6ee6525fed4a2b168dc5878604b8cd5e4f9108bb Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/56ff5381ed1b4b8725e3b1179f04dd03d072aeaf Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/0de8b9e7d02b023864630eaeaf7e91cca9cf5aa2 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/d901929468f485fcb09d4cf83d0ae6ea694c8010 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/8a5eb03d6df9f47329d2f8bb3ce1ae8e67d83943 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/e04109f533c89a4c799e754754e064ea387188a9 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/20f3285e507c659595d6ae1cedde6b5b20fca77f Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/2daa571e57ecfc774239f866bc96b7b824130402 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/d4f93b9b9def8759b95181c372e05c2df7b141f5 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/35f56e29276590e475398e6ea30b3e636890cc0c Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/6e3d4a8304291ca9d8cef32907b7259324e461db Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/f88392a5f8a8328cc93cf19901260dce52e96037 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/13da0ea4273390052d1380c8b414799e2baba22d Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/30e47959436da67e915cf07d79f10e8599a18ea1 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/09c964ca0b6118970a87c6cf06255ee25dc777cc Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/c7e4cdbe2e6bd01dc3b7d0c77ff2b61aa8223713 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/2a0418e73b206523f31e4e047cdaa96c7d227b82 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/455609ed87f7ccb35f1bea76e8847085a61c5797 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/0f60a5bb22e0f89f6fe600eab53215035cc39c7e Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/5fbe187acf08532a74d628086e05a7b10e24677d Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/9aed226806e3be21cc9560c37ee4c37190d2eb2b Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/2f697ea375f28b007ff24e9e5a2d467bb624609a Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/3d47a78612f42e7d9272490718fc67b98ddc3af9 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/999dedc4dc4e0024d2de0cb71786b5da533f00bf Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/03f0446f2a0b9952b47205a5980fc4d4a9aed464 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: [6:23:59 PM] Request /depot/16901/chunk/37cb44b4e5a564f6d2415df57528292d4458ef13 Feb 12 18:23:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:24:12 HMLGC01 SteamPrefill[5259]: [6:24:12 PM] Request /depot/16901/chunk/a06593f0208dfb0422f5aeddb9fa007e7849282e Feb 12 18:24:12 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:24:15 HMLGC01 SteamPrefill[5259]: [6:24:15 PM] Request /depot/16901/chunk/c48283957d9efbe5221d4790ac54a2435fa64463 Feb 12 18:24:15 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:24:19 HMLGC01 SteamPrefill[5259]: [6:24:19 PM] Request /depot/16901/chunk/f2cf55592d85791a915cf8ea98b3f2c4b292b2c3 Feb 12 18:24:19 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:24:21 HMLGC01 SteamPrefill[5259]: Downloading..: 66% Feb 12 18:24:51 HMLGC01 SteamPrefill[5259]: Downloading..: 68% Feb 12 18:25:21 HMLGC01 SteamPrefill[5259]: Downloading..: 70% Feb 12 18:25:27 HMLGC01 SteamPrefill[5259]: [6:25:27 PM] Request /depot/16901/chunk/b7f040b98c52ee6d4b291bb8fedaf4386098e75a Feb 12 18:25:27 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:25:51 HMLGC01 SteamPrefill[5259]: Downloading..: 72% Feb 12 18:25:56 HMLGC01 SteamPrefill[5259]: [6:25:56 PM] Request /depot/16901/chunk/7b00f43d3afbb4e0727889bd284c953b8e53ebcc Feb 12 18:25:56 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:25:59 HMLGC01 SteamPrefill[5259]: [6:25:59 PM] Request /depot/16901/chunk/b746118548bc919fa55220c7b4e178b53f23394f Feb 12 18:25:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:26:14 HMLGC01 SteamPrefill[5259]: [6:26:14 PM] Request /depot/16901/chunk/a9a9ee5f305269d70487e39d2326a097821dff58 Feb 12 18:26:14 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:26:21 HMLGC01 SteamPrefill[5259]: Downloading..: 74% Feb 12 18:26:27 HMLGC01 SteamPrefill[5259]: [6:26:27 PM] Request /depot/16901/chunk/603953aec6d7e5a8319eecd64efeb2e9654d7381 Feb 12 18:26:27 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:26:28 HMLGC01 SteamPrefill[5259]: Downloading..: 75% Feb 12 18:26:58 HMLGC01 SteamPrefill[5259]: Downloading..: 76% Feb 12 18:27:28 HMLGC01 SteamPrefill[5259]: Downloading..: 78% Feb 12 18:27:38 HMLGC01 SteamPrefill[5259]: [6:27:38 PM] Request /depot/16901/chunk/7e83250b808abeaeeabbc48f5c0e78ef31d11b29 Feb 12 18:27:38 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:27:43 HMLGC01 SteamPrefill[5259]: [6:27:43 PM] Request /depot/16901/chunk/30995db6ae0c56b51f87885974086d2f937c1a68 Feb 12 18:27:43 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:27:58 HMLGC01 SteamPrefill[5259]: Downloading..: 80% Feb 12 18:27:59 HMLGC01 SteamPrefill[5259]: [6:27:59 PM] Request /depot/16901/chunk/5fc55822e35d60dc7949e39c6a76d8d9760add63 Feb 12 18:27:59 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:28:08 HMLGC01 SteamPrefill[5259]: [6:28:08 PM] Request /depot/16901/chunk/f95fa781397fa68762c11db75addd432a903d435 Feb 12 18:28:08 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:28:12 HMLGC01 SteamPrefill[5259]: [6:28:12 PM] Request /depot/16901/chunk/d196c4ef54f3642dfd3bad918b48818eb1b68dc6 Feb 12 18:28:12 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:28:24 HMLGC01 SteamPrefill[5259]: [6:28:24 PM] Request /depot/16901/chunk/fff7c21de9c46090021712845d1c2d689213bfdf Feb 12 18:28:24 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:28:28 HMLGC01 SteamPrefill[5259]: Downloading..: 82% Feb 12 18:28:58 HMLGC01 SteamPrefill[5259]: Downloading..: 84% Feb 12 18:29:28 HMLGC01 SteamPrefill[5259]: Downloading..: 86% Feb 12 18:29:54 HMLGC01 SteamPrefill[5259]: [6:29:54 PM] Request /depot/16901/chunk/0fe40a4a9c183c36c7ef3419aee352afb9566b53 Feb 12 18:29:54 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:29:55 HMLGC01 SteamPrefill[5259]: [6:29:55 PM] Request /depot/16901/chunk/615b0ea80a63e16787adf8d2fe726e485ed882d6 Feb 12 18:29:55 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:29:58 HMLGC01 SteamPrefill[5259]: Downloading..: 88% Feb 12 18:30:02 HMLGC01 SteamPrefill[5259]: [6:30:02 PM] Request /depot/16901/chunk/0f2cd6ccdcd9d1494e849f9587a245ef3aaf1bc7 Feb 12 18:30:02 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:30:16 HMLGC01 SteamPrefill[5259]: [6:30:16 PM] Request /depot/16901/chunk/814d22f79c26aec9ed865b34d74ccc4f80d2cf95 Feb 12 18:30:16 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:30:17 HMLGC01 SteamPrefill[5259]: [6:30:17 PM] Request /depot/16901/chunk/95974acf4b95bb0e93318a9ebb1153f89235ec92 Feb 12 18:30:17 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:30:19 HMLGC01 SteamPrefill[5259]: [6:30:19 PM] Request /depot/16901/chunk/41f916b04ef5fbdba0cfd994f6dbf850153fefeb Feb 12 18:30:19 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:30:20 HMLGC01 SteamPrefill[5259]: [6:30:20 PM] Request /depot/16901/chunk/70ca65e9a373d87c8276a9ec282e0c984a8d3f1d Feb 12 18:30:20 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:30:24 HMLGC01 SteamPrefill[5259]: [6:30:24 PM] Request /depot/16901/chunk/cb92f1af4efa55da403dcb99eb8c42f2460b6208 Feb 12 18:30:24 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:30:28 HMLGC01 SteamPrefill[5259]: Downloading..: 89% Feb 12 18:30:58 HMLGC01 SteamPrefill[5259]: Downloading..: 91% Feb 12 18:30:58 HMLGC01 SteamPrefill[5259]: [6:30:58 PM] Request /depot/16901/chunk/46657101e458a5983bdc21427029594c0ab165d4 Feb 12 18:30:58 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:31:08 HMLGC01 SteamPrefill[5259]: [6:31:08 PM] Request /depot/16901/chunk/3e14c8cf34f08fccf64df90953bdf81673e60292 Feb 12 18:31:08 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:31:12 HMLGC01 SteamPrefill[5259]: [6:31:12 PM] Request /depot/16901/chunk/5ee1c33c276acc446aaab592ddf930f61069583a Feb 12 18:31:12 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException Feb 12 18:31:22 HMLGC01 SteamPrefill[5259]: [6:31:22 PM] Request /depot/16901/chunk/4fe1513e23a9db8572cd60415a4602a90729fc3e Feb 12 18:31:22 HMLGC01 SteamPrefill[5259]: failed : System.Net.Http.HttpRequestException

I get the same error as @Sysadminfromhell. Started when I update 2.4.0 as well.

Despite these errors, do the downloads succeed overall?

Despite these errors, do the downloads succeed overall?

Sometimes, sometimes some kind of error counter exceeds and the CDNs are exhausted then.
A retry then "fixes" the issue and the download normally completes.

Despite these errors, do the downloads succeed overall?

I’m in the process of rebuilding my lancache server from zero, so I’ll be able to confirm if everything’s good by the end of the day.

OK yeah if they're finishing then this might not be an issue. The core download logic didn't change significantly with the last version, but what did change is that errors are being logged to the console. So these kinds of errors were likely already happening before, they just weren't visible

Also, could I ask what region you guys are from?  Some regions tend to have higher rates of failures than others.

Also, could I ask what region you guys are from? Some regions tend to have higher rates of failures than others.

Eastern USA over here. This is an example of what I'm seeing in app.log:
[10:30:26 AM] Request /depot/977611/chunk/6f194ea2643a4e26e354a576ee1eef8fe4d37e94 failed : System.Net.Http.HttpRequestException [10:30:26 AM] Request /depot/977611/chunk/6f194ea2643a4e26e354a576ee1eef8fe4d37e94 failed : Response status code does not indicate success: 504 (Gateway Time-out).

Also, could I ask what region you guys are from? Some regions tend to have higher rates of failures than others.

I´m from germany.
[8:30:11 PM] Starting HELLDIVERS™ 2
[8:30:11 PM] Requesting available CDNs. Pool currently has 0 servers available, below the desired count of 5
[8:30:14 PM] Downloading manifests for 13 depots
[8:30:14 PM] Downloading manifest<PHONE_NUMBER>526095397 for depot 228984
[8:30:14 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:15 PM] Downloading manifest<PHONE_NUMBER>745568756 for depot 228985
[8:30:15 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:16 PM] Downloading manifest<PHONE_NUMBER>696499616 for depot 228988
[8:30:16 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:17 PM] Downloading manifest<PHONE_NUMBER>299308803 for depot 228990
[8:30:17 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:18 PM] Downloading manifest<PHONE_NUMBER>335606142 for depot 553851
[8:30:18 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:18 PM] Downloading manifest<PHONE_NUMBER>591502023 for depot 553853
[8:30:19 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:20 PM] Downloading manifest<PHONE_NUMBER>941433669 for depot 553854
[8:30:20 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:21 PM] Downloading manifest<PHONE_NUMBER>03522604 for depot 2506220
[8:30:21 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:22 PM] Downloading manifest<PHONE_NUMBER>62418282 for depot 2506210
[8:30:22 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:23 PM] Downloading manifest<PHONE_NUMBER>85454316 for depot 2506280
[8:30:23 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:24 PM] Downloading manifest<PHONE_NUMBER>511381543 for depot 2506250
[8:30:24 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:25 PM] Downloading manifest<PHONE_NUMBER>509977734 for depot 2506260
[8:30:25 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:26 PM] Downloading manifest<PHONE_NUMBER>31457626 for depot 2506240
[8:30:26 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:30:27 PM] Downloading 21.74 GiB from 44684 chunks
[8:30:27 PM] Detected Lancache server at localhost [<IP_ADDRESS>]
[8:30:27 PM] Using CDN google2.cdn.steampipe.steamcontent.com
[8:33:25 PM] Request /depot/553853/chunk/4608daa4f9096ae4ea78f13b8bfa93a06a103a97 failed : System.Net.Http.HttpRequestException
[8:33:25 PM] Request /depot/553853/chunk/4608daa4f9096ae4ea78f13b8bfa93a06a103a97 failed : Response status code does not indicate success: 502 (Bad Gateway).
[8:38:53 PM] Request /depot/553853/chunk/a9b46ebd7b9277bc09dd71449815f185d912d342 failed : System.Net.Http.HttpIOException
[8:38:53 PM] Request /depot/553853/chunk/a9b46ebd7b9277bc09dd71449815f185d912d342 failed : The response ended prematurely, with at least 19849 additional bytes expected. (ResponseEnded)
[8:46:46 PM] Request /depot/553853/chunk/d58aa0f1085c53a98a74e39c542e953b77b82f07 failed : System.Net.Http.HttpIOException
[8:46:46 PM] Request /depot/553853/chunk/d58aa0f1085c53a98a74e39c542e953b77b82f07 failed : The response ended prematurely, with at least 9408 additional bytes expected. (ResponseEnded)
[9:07:04 PM] Using CDN steampipe.akamaized.net
[9:07:07 PM] Request /depot/553853/chunk/a9b46ebd7b9277bc09dd71449815f185d912d342 failed : System.Net.Http.HttpRequestException
[9:07:07 PM] Request /depot/553853/chunk/a9b46ebd7b9277bc09dd71449815f185d912d342 failed : Response status code does not indicate success: 502 (Bad Gateway).
[9:07:07 PM] Request /depot/553853/chunk/4608daa4f9096ae4ea78f13b8bfa93a06a103a97 failed : System.Net.Http.HttpRequestException
[9:07:07 PM] Request /depot/553853/chunk/4608daa4f9096ae4ea78f13b8bfa93a06a103a97 failed : Response status code does not indicate success: 502 (Bad Gateway).
[9:07:07 PM] Request /depot/553853/chunk/d58aa0f1085c53a98a74e39c542e953b77b82f07 failed : System.Net.Http.HttpRequestException
[9:07:07 PM] Request /depot/553853/chunk/d58aa0f1085c53a98a74e39c542e953b77b82f07 failed : Response status code does not indicate success: 502 (Bad Gateway).
[9:07:11 PM] Unexpected download error : Available Steam CDN servers exhausted!  No more servers available to retry!  Try again in a few minutes  Skipping app...
[9:07:12 PM] SteamPrefill.Models.Exceptions.CdnExhaustionException: Available Steam CDN servers exhausted!  No more servers available to retry!  Try again in a few minutes
   at SteamPrefill.Handlers.Steam.CdnPool.TakeConnection()
   at SteamPrefill.Handlers.DownloadHandler.AttemptDownloadAsync(ProgressContext ctx, String taskTitle, List`1 requestsToDownload, DownloadArguments downloadArgs, Boolean forceRecache)
   at SteamPrefill.Handlers.DownloadHandler.<>c__DisplayClass6_0.<<DownloadQueuedChunksAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Spectre.Console.Progress.<>c__DisplayClass27_0.<<StartAsync>b__0>d.MoveNext() in Spectre.Console\Live\Progress\Progress.cs:line 98
--- End of stack trace from previous location ---
   at Spectre.Console.Progress.<>c__DisplayClass28_0`1.<<StartAsync>b__0>d.MoveNext() in Spectre.Console\Live\Progress\Progress.cs:line 133
--- End of stack trace from previous location ---
   at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync(Func`1 func) in Spectre.Console\Internal\DefaultExclusivityMode.cs:line 40
   at Spectre.Console.Progress.StartAsync(Func`2 action) in Spectre.Console\Live\Progress\Progress.cs:line 116
   at Spectre.Console.Progress.StartAsync(Func`2 action) in Spectre.Console\Live\Progress\Progress.cs:line 96
   at SteamPrefill.Handlers.DownloadHandler.DownloadQueuedChunksAsync(List`1 queuedRequests, DownloadArguments downloadArgs)
   at SteamPrefill.SteamManager.DownloadSingleAppAsync(UInt32 appId)
   at SteamPrefill.SteamManager.DownloadMultipleAppsAsync(Boolean downloadAllOwnedGames, Boolean prefillRecentGames, Nullable`1 prefillPopularGames)
[9:07:12 PM] Prefill complete!
[9:07:12 PM] Disconnected from Steam!
[9:07:30 PM] Starting login!
[9:07:30 PM] Connecting with CellId: 87
[9:07:31 PM] Connected to Steam!
[9:07:31 PM] Logged into Steam
[9:07:32 PM] Using CM <IP_ADDRESS> - Germany - Frankfurt am Main
[9:07:32 PM] Loaded account licenses                                  00.3022
[9:07:32 PM] Steam session initialization complete!                   01.6693
[9:07:32 PM] Loaded metadata for 14 apps                              00.5969
[9:07:33 PM] Loaded metadata for 14 apps                              00.5873
[9:07:33 PM] Starting HELLDIVERS™ 2
[9:07:33 PM] Requesting available CDNs. Pool currently has 0 servers available, below the desired count of 5
[9:07:36 PM] Downloading manifests for 13 depots
[9:07:36 PM] Downloading 21.74 GiB from 44684 chunks
[9:07:36 PM] Detected Lancache server at localhost [<IP_ADDRESS>]
[9:07:36 PM] Using CDN steampipe.akamaized.net
[9:13:06 PM] Finished in 05:30.24 - 565.58 Mbit/s
[9:13:06 PM] Prefill complete!
[9:13:06 PM] Disconnected from Steam!
[12:00:38 AM] Starting login!
[12:00:51 AM] Connecting with CellId: 87
[12:01:21 AM] Already disconnected from Steam
[12:01:25 AM] SteamPrefill.Models.Exceptions.SteamConnectionException: Timeout connecting to Steam...  Try again in a few moments
   at SteamPrefill.Handlers.Steam.Steam3Session.ConnectToSteam()
   at SteamPrefill.Handlers.Steam.Steam3Session.<>c__DisplayClass16_0.<<LoginToSteamAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Spectre.Console.Status.<>c__DisplayClass16_0.<<StartAsync>b__0>d.MoveNext() in Spectre.Console\Live\Status\Status.cs:line 79
--- End of stack trace from previous location ---
   at Spectre.Console.Status.<>c__DisplayClass17_0`1.<<StartAsync>b__0>d.MoveNext() in Spectre.Console\Live\Status\Status.cs:line 120
--- End of stack trace from previous location ---
   at Spectre.Console.Progress.<>c__DisplayClass28_0`1.<<StartAsync>b__0>d.MoveNext() in Spectre.Console\Live\Progress\Progress.cs:line 133
--- End of stack trace from previous location ---
   at Spectre.Console.Internal.DefaultExclusivityMode.RunAsync(Func`1 func) in Spectre.Console\Internal\DefaultExclusivityMode.cs:line 40
   at Spectre.Console.Progress.StartAsync(Func`2 action) in Spectre.Console\Live\Progress\Progress.cs:line 116
   at Spectre.Console.Status.StartAsync(String status, Func`2 func) in Spectre.Console\Live\Status\Status.cs:line 117
   at Spectre.Console.Status.StartAsync(String status, Func`2 action) in Spectre.Console\Live\Status\Status.cs:line 77
   at SteamPrefill.Handlers.Steam.Steam3Session.LoginToSteamAsync()
   at SteamPrefill.SteamManager.InitializeAsync()
   at SteamPrefill.CliCommands.PrefillCommand.ExecuteAsync(IConsole console)
   at CliFx.CliApplication.RunAsync(ApplicationSchema applicationSchema, CommandInput commandInput) in C:\Users\Tim\Dropbox\Programming\Lancache-Prefills\External\CliFx\CliFx\CliApplication.cs:line 148
   at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments, IReadOnlyDictionary`2 environmentVariables) in C:\Users\Tim\Dropbox\Programming\Lancache-Prefills\External\CliFx\CliFx\CliApplication.cs:line 190
   at CliFx.CliApplication.RunAsync(IReadOnlyList`1 commandLineArguments) in C:\Users\Tim\Dropbox\Programming\Lancache-Prefills\External\CliFx\CliFx\CliApplication.cs:line 202
   at SteamPrefill.Program.Main()

I also see a lot of this messages:
'''
[4:20:28 AM] Starting Lethal Company
[4:20:28 AM] Requesting available CDNs. Pool currently has 4 servers available, below the desired count of 5
'''

OK yeah if they're finishing then this might not be an issue. The core download logic didn't change significantly with the last version, but what did change is that errors are being logged to the console. So these kinds of errors were likely already happening before, they just weren't visible

I think you're right about what's happening. Everything works just fine when clients are downloading from the lancache server later. There are no cache misses or anything. So, at least for me, everything is good.

Sooo i tested in Version 2.3.0 and 2.2.0 also 2.5.0 and in all the versions i have this error...

When you're running the newest version, do the downloads eventually finish successfully?

Today, the same error

OK yeah if they're finishing then this might not be an issue. The core download logic didn't change significantly with the last version, but what did change is that errors are being logged to the console. So these kinds of errors were likely already happening before, they just weren't visible

I think you're right about what's happening. Everything works just fine when clients are downloading from the lancache server later. There are no cache misses or anything. So, at least for me, everything is good.

The interesting is, in my case the download from the cache server fails with "no internet connection" ore has 1kb/s ---> what i think is a fail to. But the prefill downloads with 50mb/s (what my internet speed is)

I testet it with diferent systems, clean lxc ubuntu 23, kali linux vm and other. All have the same problem

I found my error, the lancache server dosnt had permissions to write in the cache... LuL
So i changed the path... and wow it works


New linter: SpacesForIndentation

Disabled by default, since tabs are acceptable in HAML, but folks may
want to enable this to enforce spaces-only.

Thanks for the pull request @gburt.
I've merged your change in 7707e61 and changed it to support specifying which type of indent character to enforce in 134c4146e2. The name has been changed to Indentation to reflect this more generic behavior.


[Proposal] Schemaless JSON fields for Eloquent

The Problem
Consider the following scenario. You need to store semi-structured data for one or more attributes of an Eloquent model. Since these attributes are only relevant to a subset of the rows in your database, you don’t want to create a new field for every single one of those attributes in your table.
To illustrate the problem, let's imagine we have a simple database table that stores business cards:
business_cards
  first_name
  last_name
  email

If we wanted to have the same flexibility as a format like vCard, which has a multitude of fields and even allows for custom fields, we would have to create a new column for each new field:
business_cards
  first_name
  last_name
  email
  x_gender
  x_twitter
  ...

These requirements would make a schemaless solution like MongoDB ideal. However, in the context of a Laravel application with existing models and in which the data is primarily relational, this isn't a viable solution.
While it is already possible to use a TEXT field and manually serialize data to JSON when saving to it (and then deserialize it when reading from it), this approach has two shortcomings:

It is inefficient and inelegant
It doesn't use the native json datatype in PostgreSQL and therefore it cannot be indexed

The Solution
The proposed solution consists in adding new methods to interact with JSON fields to Eloquent.
There are at least three aspects to consider to make this work:

The schema builder would need a new json field type. This field would be turned into a json datatype when using PostgreSQL and for all other databases it would simply become a TEXT field:

$table->json('social_profiles');

There should be a way of defining the list of JSON fields in the model (in the same way as getDates()):

getJsonFields() {
  return array('social_profiles');
}

Lastly and most importantly, these fields would need to provide a set of methods to interact with their underlying JSON object:

$business_card->social_profiles->set('twitter', 'laravelPHP');
// or
$business_card->social_profiles->twitter = 'laravelPHP';

$business_card->social_profiles->get('twitter');
// or
$business_card->social_profiles->twitter; // => 'laravelPHP'

$business_card->social_profiles->has('xing'); // => FALSE

The JSON field, in this case social_profiles, would be deserialized on demand when it is first accessed and serialized – if it was changed – when the model is saved.

I’d be interested in hearing your thoughts on the suggested implementation. Also, if you know of any similar implementations in other frameworks, I'd be happy to hear about them.

See https://github.com/darrylkuhn/dialect

Unfortunately darrylkuhn/dialect conflicts with some other packages I use due to what I presume is the altered Eloquent results.
It'd be ideal if querying JSON fields was possible with Eloquent out of the box, my short lived use of darrylkuhn/dialect did drastically simplify my code, it's a shame it conflicts too much with the other needs of my application.
I hope this feature is considered in the near future for Laravel 5.3 onwards.

@AdrianB93 do you have any specific examples of packages that are conflicting with dialect?

@raphaelsaunier, VentureCraft/revisionable is the most notable example, there were other conflicts but I don't remember which ones they were. I ended up choosing to make an accessor in my model for some JSON field values using json_decode and placing it in a Collection.

You can already use JSON fields, casting them as an array. Eloquent does json_encode/json_decode for you.
https://laravel.com/docs/master/eloquent-mutators#attribute-casting

You can already use JSON fields, casting them as an array. Eloquent does json_encode/json_decode for you.

That wasn't a feature yet when this issue was opened though :p

:+1


User is unable to get the updated version in the server

Hi,
I am using angular 4 version of this project. I compile and publish it to a Azure VM which is my web server. I notice that my user will not automatically see the latest version. Only way for user to get the new version is to manually clear cache in chrome and open the URL again.
Please enlighten me. Thank you.

See commit 82afbb5781165081e6167e4679a89ed1caeba8b6

Closing as fixed via 82afbb5, hope that helps fix your issue @helloneoberg !


Support all float dtypes in F.sigmoid_cross_entropy

This PR follows #4582 to support all float dtypes in F.sigmoid_cross_entropy.
I found that it has relatively large error in forward computation when it computes FP64 data with CuPy. It seems that this line with dtype argument would cause the error between numpy and CuPy. I will check it later.

I got the cause of the problem. When dtype=numpy.float64 is supplied to cupy.divide, cupy.true_divide's bb->d routine is actually selected for its operation in Py3, consequently the arguments are casted before division.
In numpy, the same problem had ever caused and seems to be fixed in https://github.com/numpy/numpy/pull/9469 .

Jenkins, test this please.

Jenkins CI test (for commit 0e40ed4907a55ff31253b68fd9002ee4edc049bb, target branch master) succeeded!

Fixed the large error problem with CuPy FP64.

Jenkins, test this please.

Jenkins CI test (for commit f5b6de3f9c0296f64a78b92248bb20f218bbe1c3, target branch master) succeeded!

Jenkins, test this please.

LGTM!


Bug: Not updating slug after title has changed

Issue Summary
When starting a new draft before publishing, the slug name is not changed when the title of the article has changed.
Steps to Reproduce

Start a new draft
Add a title
Save the draft
Check the slug by pressing the cog wheel in the lower left hand corner and viewing the slug field.
Change the title in the document.
Save the draft.
Verify the slug has not changed.

This is a bug because if the title has changed before it has been published the slug should also be updated.
Technical details

Ghost Version: master - latest commit:  26548a734a3572b656eabd51e9a848a4cf83df21
Client OS: OSX 10.10
Server OS: Ubuntu 12.04
Node Version: v0.12.3
Browser: Safari [Version 8.0.6 (10600.6.3)]

This sounds like a duplicate of #5062 , and I'm pretty sure there's a PR in to fix this.

@acburdine Thank you, I was looking through the open issues and glossed over that issue because the language it used was off. I was going to fork and put a request in to fix this, but it seems like it is going to get resolved.

Closing as a duplicate of #5062

There is a pr but needs further discussion. Ill continue with it once I am
back to the real world (I am on hols)
On Tuesday, May 19, 2015, Hannah Wolfe<EMAIL_ADDRESS>wrote:

Closed #5289 https://github.com/TryGhost/Ghost/issues/5289.
—
Reply to this email directly or view it on GitHub
https://github.com/TryGhost/Ghost/issues/5289#event-308615293.

@dgonzalez can you reference the PR? Maybe I could help.

#5216

https://github.com/TryGhost/Ghost/pull/5216
2015-05-21 17:43 GMT+01:00 Gabriel Csapo<EMAIL_ADDRESS>
@dgonzalez https://github.com/dgonzalez can you reference the PR? Maybe
I could help.
—
Reply to this email directly or view it on GitHub
https://github.com/TryGhost/Ghost/issues/5289#issuecomment-104349167.