Failed to create Engine for SSD-Mobilenet on DLA core

python3: ../rtExt/dla/native/dlaExecuteRunner.cpp:123: virtual void nvinfer1::rt::dla::DLANativeRunner::allocateResources(const nvinfer1::rt::CommonContext&): Assertion `context.dlaContext->getDLACore() >= 0 && context.dlaContext->getDLACore() < mCore.numEngines()' failed.
Traceback (most recent call last):
File "code/main.py", line 328, in
main()
File "code/main.py", line 318, in main
launch_handle_generate_engine(benchmark_name, benchmark_conf, need_gpu, need_dla)
File "code/main.py", line 80, in launch_handle_generate_engine
raise RuntimeError("Building engines failed!")
RuntimeError: Building engines failed!
Makefile:298: recipe for target 'generate_engines' failed
make: *** [generate_engines] Error 1

@sandip761 Could you try the suggested solution here: https://github.com/mlperf/inference_results_v0.5/issues/23#issuecomment-597358583 ? Let me know if that doesn't work. Thanks

I am able to create engine dla core , while running for performance i got this errror.
[04/25/2020-16:40:39] [E] [TRT] ../rtExt/dla/native/dlaExecuteRunner.cpp (71) - DLA Error in execute: 7 (Input tensor register failed.)
[04/25/2020-16:40:39] [E] [TRT] FAILED_EXECUTION: std::exception
NVMEDIA_DLA : 1599, ERROR: registerTensorBuffer failed
[04/25/2020-16:40:39] [E] [TRT] ../rtExt/dla/native/dlaExecuteRunner.cpp (71) - DLA Error in execute: 7 (Input tensor register failed.)
[04/25/2020-16:40:39] [E] [TRT] FAILED_EXECUTION: std::exception
NVMEDIA_DLA : 1599, ERROR: registerTensorBuffer failed
[04/25/2020-16:40:39] [E] [TRT] ../rtExt/dla/native/dlaExecuteRunner.cpp (71) - DLA Error in execute: 7 (Input tensor register failed.)
[04/25/2020-16:40:39] [E] [TRT] FAILED_EXECUTION: std::exception
NVMEDIA_DLA : 1599, ERROR: registerTensorBuffer failed
[04/25/2020-16:40:39] [E] [TRT] ../rtExt/dla/native/dlaExecuteRunner.cpp (71) - DLA Error in execute: 7 (Input tensor register failed.)
[04/25/2020-16:40:39] [E] [TRT] FAILED_EXECUTION: std::exception
NVMEDIA_DLA : 1599, ERROR: registerTensorBuffer failed
[04/25/2020-16:40:39] [E] [TRT] ../rtExt/dla/native/dlaExecuteRunner.cpp (71) - DLA Error in execute: 7 (Input tensor register failed.)
[04/25/2020-16:40:39] [E] [TRT] FAILED_EXECUTION: std::exception
[04/25/2020-16:40:39] [I] Average over 10 runs is 0.393837 ms (host walltime is 0.453828 ms, 99% percentile time is 0.395264).
trtexec: ../rtExt/dla/nvmItem.cpp:22: nvinfer1::rt::dla::NvmItem::~NvmItem(): Assertion `!mCudaMemory || !mNvmTensor' failed.
Aborted (core dumped)

@sandip761 You built the DLA engine with BS=8 but ran it with BS=4. DLA does not support dynamic batch size.


malloc(): unsorted double linked list corrupted Fatal Python error: Aborted

The process is getting stopped after throwing this error.

I've seen sometimes too.  I think full stacktrace shows some chroma db issue.

@RohanKaran @pseudotensor could you please give the steps to reproduce this?

@Mathanraj-Sharma There's no easy repro AFAIK, but maybe he sees consistently.

Closing this since there are no new activities in this ticket, please feel free to reopen if required


Apple Mail Module 3.0.0: signature no longer inserted into Composed messages

In older versions of this plug-in, using the Apple Mail "Compose E-mail" action would open a new mail message with my default signature already inserted into the body of the message. After upgrading to B69 (64-bit) and Apple Mail Module 3.0.0, new messages are opened completely blank. It would be nice to restore the signature insertion if possible.

Unfortunately Apple has restricted the use of Apple Mail for 'security' reasons. As such, we've had to disable the Apple Mail plugin. If Apple ever allow access to the Mail settings in the future, we can revisit this.


status of pipelinerun changed to PipelineRunTimeout several hours after pipelinerun ended successfully

Expected Behavior
Status if pipelinerun should not change after several hours.
Actual Behavior
Given a pipeline with some tasks. If all tasks in the pipeline are executed sucessfully then the pipeline status should be Success.
But after several hours the status is updated to PipelineRunTimeout (seems to be a regression in v.0.47.0):
{"severity":"info","timestamp":"2023-05-25T09:19:43.378Z","logger":"tekton-pipelines-controller","caller":"pipelinerun/pipelinerun.go:878","message":"Creating a new TaskRun object XXX-pipeline-run-sqbpb-bump-version for pipeline task bump-version","commit":"8e8c163","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.pipelinerun.Reconciler","knative.dev/kind":"tekton.dev.PipelineRun","knative.dev/traceid":"24862cdd-63ac-46f0-ab45-531b8f2bf6f9","knative.dev/key":"tekton-pipelines/XXX-pipeline-run-sqbpb"}

...

{"severity":"info","timestamp":"2023-05-25T09:19:43.396Z","logger":"tekton-pipelines-controller.event-broadcaster","caller":"record/event.go:285","message":"Event(v1.ObjectReference{Kind:\"TaskRun\", Namespace:\"tekton-pipelines\", Name:\"XXX-pipeline-run-sqbpb-bump-version\", UID:\"05ba2a79-d2a3-42db-b393-31bc206823d5\", APIVersion:\"tekton.dev/v1beta1\", ResourceVersion:\"289729588\", FieldPath:\"\"}): type: 'Normal' reason: 'Started' ","commit":"8e8c163"}

...

{"severity":"info","timestamp":"2023-05-25T09:19:56.364Z","logger":"tekton-pipelines-controller","caller":"taskrun/taskrun.go:530","message":"Successfully reconciled taskrun XXX-pipeline-run-sqbpb-bump-version/tekton-pipelines with status: &apis.Condition{Type:\"Succeeded\", Status:\"True\", Severity:\"\", LastTransitionTime:apis.VolatileTime{Inner:time.Date(2023, time.May, 25, 9, 19, 56, 364699830, time.Local)}, Reason:\"Succeeded\", Message:\"All Steps have completed executing\"}","commit":"8e8c163","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"0a68760b-f7f4-47f1-940e-66b53055574d","knative.dev/key":"tekton-pipelines/XXX-pipeline-run-sqbpb-bump-version"}

several hours later
{"severity":"info","timestamp":"2023-05-25T13:02:26.816Z","logger":"tekton-pipelines-controller","caller":"pipelinerun/timeout.go:112","message":"cancelling TaskRun XXX-pipeline-run-sqbpb-bump-version for timeout","commit":"8e8c163","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.pipelinerun.Reconciler","knative.dev/kind":"tekton.dev.PipelineRun","knative.dev/traceid":"ab2b052a-f937-4780-ad8f-efdb350e053d","knative.dev/key":"tekton-pipelines/XXX-pipeline-run-sqbpb"}

Problem occurred after upgrading from v0.41.0 to v0.47.0
Steps to Reproduce the Problem

create pipeline with a some tasks
run pipeline
wait several hours -> pipelinerun status will change from "Successful" to "PipelineRunTimeout"

Additional Info

Kubernetes version:
Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.0", GitCommit:"b46a3f887ca979b1a5d14fd39cb1af43e7e5d12d", GitTreeState:"clean", BuildDate:"2022-12-08T19:58:30Z", GoVersion:"go1.19.4", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.9", GitCommit:"9710807c82740b9799453677c977758becf0acbb", GitTreeState:"clean", BuildDate:"2022-12-08T10:08:06Z", GoVersion:"go1.18.9", Compiler:"gc", Platform:"linux/amd64"}

Tekton Pipeline version:
Output of tkn version or kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'

Client version: 0.31.0
Pipeline version: v0.47.0
Triggers version: v0.24.0
Dashboard version: v0.35.0

We need to cut v0.47.1 with the patch - https://github.com/tektoncd/pipeline/pull/6634, https://github.com/tektoncd/pipeline/pull/6622
For now, you can upgrade to v0.48 where this issue is fixed

:+1:


test: fix the flaky of RemoveListenerAfterInPlaceUpdate

Commit Message: test: fix the flaky of RemoveListenerAfterInPlaceUpdate
Additional Description:
The listener sockets are closed async, so using a while loop to detect a connect success or not.
Risk Level: low
Testing: flaky test
Fixes #22348

/retest


hear works on Terminal, but not on Alacritty: NSSpeechRecognitionUsageDescription key missing.

When I run 'hear' from Alacritty it does not ask access to speech recognition and return 'zsh: abort'.
Console.app:

ERROR: This app has crashed because it attempted to access privacy-sensitive data without a usage description.  The app's Info.plist must contain an NSSpeechRecognitionUsageDescription key with a string value explaining to the user how the app uses this data.

On the other hand, when I tried to run 'hear' using Terminal.app, it worked perfectly. Terminal asked for access to speech recognition and started outputting the transcribed text.
Apple documentation says that "You must include the NSSpeechRecognitionUsageDescription key in your app’s Info.plist file. If this key is not present, your app will crash when it attempts to request authorization or use the APIs of the Speech framework."(1) Because "This key is required if your app uses APIs that send user data to Apple’s speech recognition servers."(2)
I added to /Applications/Alacritty.app/Contents/Info.plist:
<key>NSSpeechRecognitionUsageDescription</key>
<string>An application in Alacritty would like to access speech recognition.</string>

Now 'hear' exit with:

Error: Speech recognition authorization not determined
https://github.com/sveinbjornt/hear/blob/e3247563695cc38bc05953c706c909910fe242d3/src/Hear.m#L114

Does anyone know how to solve this issue? Or have any suggestion?
Thanks!

I tried adding NSSpeechRecognitionUsageDescription to Alacritty's Info.plist, and was able to reproduce your result.
However, launching Alacritty from the command line in iTerm or Terminal.app works!:
$ /Applications/Alacritty.app/Contents/MacOS/alacritty

I haven't figured out why but this lets me use hear directly Alacritty
what I did is I launched skhd through alacritty and had it run a script that uses hear. that that point i was asked by the os (13.x) if I wanted to let Alacritty use the speech recognition and I did. Now I can directly call hear in Alacritty and it works fine.

I encountered the same zsh abort issue when running hear in the vscode terminal. Similar to Alacritty, speech recognition permission doesn't list vscode but I don't have a way to request that permission.


Cannot connect as partner

When attempting to connect as a partner, we get a generic error. Do you have any further info on how this process is meant to work?

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4ebdb9f0-76a5-eebc-4cd7-95f5407ed029
Version Independent ID: 05abf201-e31c-08c0-a7fc-8fd46202e8b0
Content: Microsoft 365 usage analytics - Microsoft 365 admin
Content Source: microsoft-365/admin/usage-analytics/usage-analytics.md
Service: o365-administration
GitHub Login: @Sirkkuw
Microsoft Alias: sirkkuw

@BlueOdyssey Thank you for your feedback. As a partner, if you have delegated admin rights then you connect to the template app on behalf of your customer.
If the access is present, Then I would start which checking if Microsoft usage analytics is enabled and the Template app is initiated in power BI. More information can be found in the document -https://docs.microsoft.com/en-AU/microsoft-365/admin/usage-analytics/enable-usage-analytics?view=o365-worldwide
Hope this helps!
Thanks
Sri


[YUNIKORN-1982] Enable manual triggering for GitHub CI

What is this PR for?
Allow running manually from the github ui.
Everyone can manually run the CI on their fork.
github docs

https://docs.github.com/en/actions/using-workflows/manually-running-a-workflow
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_dispatch

This will not affect ANY github action we are running.
What type of PR is it?

[x] - Improvement

What is the Jira issue?
YUNIKORN-1982

@wilfred-s
We only manual trigger pre-commit hook.
Please Take a look. 🙏

+1 LGTM


.fasta icon (and related)

FASTA format (and its relatives) are used heavily in computational biology. Could we please get a DNA symbol for it?
Affected formats are:

.fasta
.fa
.fastq
.fq
.fna
.faa
.ffn
.frn
.sam

Not a problem! 😉 Would this be okay...?

Perfect!

On Jul 1, 2018, at 12:15 AM, John Gardner<EMAIL_ADDRESS>wrote:
Not a problem! 😉 Would this be okay...?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Awesome. Thanks for letting me know! 😁
I'll probably cut a release in the next couple of days, so I'll regenerate the icon files then. 😉


Visual Studio 2017 - Unit tests are not showing in Test Explorer

In Visual Studio 2017, unit tests are not showing in test explorer. Rebuilding the project and clicking on "Run All" as no effect.
This isn't an issue in Visual Studio 2015.
Thanks

We probably need to update some tooling nuget package version. Can somebody check the ones used on ASP.NET Core MVC on the master branch?

they dont seem to work in 2015 for me anymore either :|

I stumbled upon this issue. I think it's been fixed, because I can see the tests in the Test Explorer in VS 2015.

I also think this has been fixed. The tests appear in VS2017 (15.7.4) as well.

Please reopen if you can still repro


May I ask if there is a corresponding paper available

The early access paper is https://ieeexplore.ieee.org/abstract/document/10632058. please cite it if helpful :)

thank you very much


Fix warnings / deprecations etc. in relation to former Akka 2.5 support

Some things could not properly fixed to keep compatibility with Akka 2.5. Since we don't care anymore about 2.5, we can clean things up.

stream cancellation handling (getting causes in onDownstreamFinish)

Mhaha, ghosts from the past.
Does sound more like a 1.1 thing though?

Does sound more like a 1.1 thing though?

Yes, these changes should be in 1.1


docs: Update configuration.md, added sbom generation flag

Description
Related issues

Close #XXX

Remove this section if you don't have related PRs.
Checklist

[ ] I've read the guidelines for contributing to this repository.
[ ] I've added tests that prove my fix is effective or that my feature works.
[ ] I've updated the documentation with the relevant information (if needed).
[ ] I've added usage information (if the PR introduces new options)
[ ] I've included a "before" and "after" example to the description (if the PR is a user interface change).

@Talbalash-legit please sign CLA


⚠ SkipTheCommericals has degraded performance

In d50217a, SkipTheCommericals ($STC) experienced degraded performance:

HTTP code: 200
Response time: 3845 ms

Resolved: SkipTheCommericals performance has improved in 77a778f.


clippy: allow macro_metavars_in_unsafe

(part of #2487)
Problem
https://rust-lang.github.io/rust-clippy/master/index.html#/macro_metavars_in_unsafe
Summary of Changes
it works for years. I guess we can just allow it 🤔

oh... it's a new lint. need to happen in the bumping PR 🫠


Kafka index service use a lot of direct memory during segment publish

Hi, all
I found a problem when i use kafka index service. The task process uses a lot of direct memory when publishing segments, almost 6x than ingesting data stage.
I'm using druid-0.13.0.
At ingesting time, RES of the task is 1.647g.
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
29321 root      20   0 12.691g 1.647g  30952 S   8.7  5.3   3:52.96 java

And the heap 、non-heap、direct are 221M、120M and 864M
Memory                                     used           total         max            usage         GC
heap                                       221M           502M          2048M          10.80%        gc.g1_young_generation.count                      134
g1_eden_space                              43M            248M          -1             17.34%        gc.g1_young_generation.time(ms)                   1146
g1_survivor_space                          38M            38M           -1             100.00%       gc.g1_old_generation.count                        0
g1_old_gen                                 140M           216M          2048M          6.85%         gc.g1_old_generation.time(ms)                     0
nonheap                                    120M           124M          -1             96.74%
code_cache                                 40M            42M           240M           17.05%
metaspace                                  70M            72M           -1             96.62%
compressed_class_space                     9M             9M            1024M          0.91%
direct                                     864M           864M          -              100.00%

But at publishing time, RES of the task is 6.046g
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
29321 root      20   0 17.002g 6.046g 109592 S 142.7 19.3   5:31.43 java

And the heap 、non-heap、direct are 266M、124M and 5204M
Memory                                     used           total         max            usage         GC
heap                                       266M           502M          2048M          13.01%        gc.g1_young_generation.count                      187
g1_eden_space                              13M            198M          -1             6.57%         gc.g1_young_generation.time(ms)                   1802
g1_survivor_space                          4M             4M            -1             100.00%       gc.g1_old_generation.count                        0
g1_old_gen                                 249M           300M          2048M          12.18%        gc.g1_old_generation.time(ms)                     0
nonheap                                    124M           134M          -1             92.48%
code_cache                                 44M            44M           240M           18.47%
metaspace                                  70M            78M           -1             89.42%
compressed_class_space                     9M             10M           1024M          0.91%
direct                                     5204M          5204M         -              100.00%

The direct memory increase 864M to 5204M when publishing begins. How can i control the usage of direct memory?
Any suggestion? Thanks!
middleManager runtime.properties
druid.service=druid/middleManager
druid.port=8091

# Number of tasks per middleManager
druid.worker.capacity=10

# Task launch parameters
druid.indexer.runner.javaOpts=-server -Xmx2g -XX:MaxDirectMemorySize=10240g -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
druid.indexer.task.baseTaskDir=var/druid/task

# HTTP server threads
druid.server.http.numThreads=50

# Processing threads and buffers on Peons
druid.indexer.fork.property.druid.processing.buffer.sizeBytes=100000000
druid.indexer.fork.property.druid.processing.numThreads=7
druid.indexer.task.restoreTasksOnRestart=true

# Hadoop indexing
druid.indexer.task.hadoopWorkingPath=var/druid/hadoop-tmp
druid.indexer.task.defaultHadoopCoordinates=["org.apache.hadoop:hadoop-client:2.7.3"]

Datasource has 35 dimensions and 45 metrics. The spec file
{
    "type": "kafka",
    "dataSchema": {
        "dataSource": "MY_DATASOURCE_DAY",
        "parser": {
            "type": "string",
            "parseSpec": {
                "format": "json",
                "timestampSpec": {
                    "column": "timestamp",
                    "format": "millis"
                },
                "dimensionsSpec": {
                    "dimensions": [
			"mp_id",
			"application_id",
			"instance_id",
			"action_name",
			"message_type",
			"from_path",
			"path",
			"open_path",
			"close_path",
			"scene",
			"country_id",
			"region_id",
			"city_id",
			"carrier_id",
			"error_message",
			"error_filename",
			"request_method",
			"host",
			"uri",
			"network_type",
			"wechat_version",
			"route_chain",
			"uid",
			"http_code",
			"system",
			"ip",
			"device_type",
			"agreement_id",
			"protocol",
                        "error_type",
                        "error_detail",
                        "custom_code",
			"api_name",
                        "opera_name",
                        "offset_left",
                        "offset_top",
			"agent_version"
                    ]
                }
            }
        },
        "metricsSpec": [
            {
                "name": "server_count",
                "fieldName": "server_count",
                "type": "longSum"
            },
            {
                "name": "quit_count",
                "fieldName": "quit_count",
                "type": "longSum"
            },
            {
                "name": "on_ready",
                "fieldName": "on_ready",
                "type": "doubleSum"
            },
            {
                "name": "custom_time",
                "fieldName": "custom_time",
                "type": "longSum"
            },
            {
                "name": "first_response_time",
                "fieldName": "first_response_time",
                "type": "doubleSum"
            },
            {
                "name": "response_time",
                "fieldName": "response_time",
                "type": "doubleSum"
            },
            {
                "name": "application_server_time",
                "fieldName": "application_server_time",
                "type": "doubleSum"
            },
            {
                "name": "network_time",
                "fieldName": "network_time",
                "type": "doubleSum"
            },
            {
                "name": "callback_time",
                "fieldName": "callback_time",
                "type": "longSum"
            },
            {
                "name": "bytes_sent",
                "fieldName": "bytes_sent",
                "type": "longSum"
            },
            {
                "name": "bytes_received",
                "fieldName": "bytes_received",
                "type": "longSum"
            },
            {
                "name": "msg_error_pv",
                "fieldName": "msg_error_pv",
                "type": "longSum"
            },
            {
                "name": "file_error_pv",
                "fieldName": "file_error_pv",
                "type": "longSum"
            },
            {
                "name": "count",
                "fieldName": "count",
                "type": "longSum"
            },
            {
                "name": "on_ready_count",
                "fieldName": "on_ready_count",
                "type": "longSum"
            },
            {
                "name": "open_count",
                "fieldName": "open_count",
                "type": "longSum"
            },
            {
                "name": "net_count",
                "fieldName": "net_count",
                "type": "longSum"
            },
            {
                "name": "net_error_count",
                "fieldName": "net_error_count",
                "type": "longSum"
            },
            {
                "name": "js_error_count",
                "fieldName": "js_error_count",
                "type": "longSum"
            },
            {
                "name": "slow_count",
                "fieldName": "slow_count",
                "type": "longSum"
            },
            {
                "name": "net_slow_count",
                "fieldName": "net_slow_count",
                "type": "longSum"
            },
            {
                "name": "custom_success_count",
                "fieldName": "custom_success_count",
                "type": "longSum"
            },
            {
                "name": "uv",
                "fieldName": "uid",
                "type": "thetaSketch"
            },
            {
                "name": "api_success_count",
                "fieldName": "api_success_count",
                "type": "longSum"
            },
            {
                "name": "api_fail_count",
                "fieldName": "api_fail_count",
                "type": "longSum"
            },
            {
                "name": "api_count",
                "fieldName": "api_count",
                "type": "longSum"
            },
            {
                "name": "api_cancel_count",
                "fieldName": "api_cancel_count",
                "type": "longSum"
            },
            {
                "type": "longSum",
                "name": "opera_time",
                "fieldName": "opera_time",
                "expression": null
            },
            {
                "type": "longSum",
                "name": "opera_request_time",
                "fieldName": "opera_request_time",
                "expression": null
            },
            {
                "type": "longSum",
                "name": "opera_server_time",
                "fieldName": "opera_server_time",
                "expression": null
            },
            {
                "type": "longSum",
                "name": "opera_count",
                "fieldName": "opera_count",
                "expression": null
            },
            {
                "type": "longSum",
                "name": "opera_request_count",
                "fieldName": "opera_request_count",
                "expression": null
            },
            {
                "type": "longSum",
                "name": "opera_success_count",
                "fieldName": "opera_success_count",
                "expression": null
            },
            {
                "type": "longSum",
                "name": "opera_fail_count",
                "fieldName": "opera_fail_count",
                "expression": null
            },
            {
                "type": "longSum",
                "name": "opera_slow_count",
                "fieldName": "opera_slow_count",
                "expression": null
            },
            {
                "type": "longSum",
                "name": "opera_server_count",
                "fieldName": "opera_server_count",
                "expression": null
            },
            {
                "type": "longSum",
                "name": "page_net_error_count",
                "fieldName": "page_net_error_count",
                "expression": null
            },
	    {
	        "type" : "quantilesDoublesSketch",
	        "name" : "on_ready_sketch",
	        "fieldName" : "on_ready",
	        "k": 256
	    },
	    {
	        "type" : "quantilesDoublesSketch",
	        "name" : "first_response_time_sketch",
	        "fieldName" : "first_response_time",
	        "k": 256
	    },
	    {
	        "type" : "quantilesDoublesSketch",
	        "name" : "response_time_sketch",
	        "fieldName" : "response_time",
	        "k": 256
	    },
	    {
	        "type" : "quantilesDoublesSketch",
	        "name" : "network_time_sketch",
	        "fieldName" : "network_time",
	        "k": 256
	    },
	    {
	        "type" : "quantilesDoublesSketch",
	        "name" : "application_server_time_sketch",
	        "fieldName" : "application_server_time",
	        "k": 256
	    },
            {
                "type": "quantilesDoublesSketch",
                "name": "opera_time_sketch",
                "fieldName": "opera_time",
                "k": 256
            },
            {
                "type": "quantilesDoublesSketch",
                "name": "opera_request_time_sketch",
                "fieldName": "opera_request_time",
                "k": 256
            },
            {
                "type": "quantilesDoublesSketch",
                "name": "opera_server_time_sketch",
                "fieldName": "opera_server_time",
                "k": 256
            }
        ],
        "granularitySpec": {
            "type": "uniform",
            "segmentGranularity": "DAY",
            "queryGranularity": "DAY"
        }
    },
    "tuningConfig": {
        "type": "kafka",
        "maxRowsPerSegment": 2500000,
	"resetOffsetAutomatically": true,
        "targetPartitionSize": 2500000
    },
    "ioConfig": {
        "topic": "drd-mp-webchat-applet",
        "consumerProperties": {
            "bootstrap.servers": "kafka-server:9092"
        },
        "taskCount": 1,
        "taskDuration": "PT1H",
        "replicas": 1,
	"useEarliestOffset": true
    }
}

I found a lot of Allocating new littleEndByteBuf operations in the log.
Finally, allocated 333023 littleEndByteBuf and the usage of direct memory arrived 18GB, and OS killed this task.
The ingested data was very small, just 2GB in six hours, and all the dimensions have very low cardinality.
Why the task need so many direct memory?
@himanshug
Can you give me some advices?  Thank you so much!
2019-06-14T06:24:34,445 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,020]
2019-06-14T06:24:34,445 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,021]
2019-06-14T06:24:34,445 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,022]
2019-06-14T06:24:34,445 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,023]
2019-06-14T06:24:34,445 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,024]
2019-06-14T06:24:34,445 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,025]
2019-06-14T06:24:34,446 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,026]
2019-06-14T06:24:34,446 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,027]
2019-06-14T06:24:34,446 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,028]
2019-06-14T06:24:34,446 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,029]

Unfortunately, It is very hard to offer concrete advice on these things, I can give some general points for you to further investigate.

OS killed this task

I am assuming that means linux OOM killer killed the process. If yes, then it probably can't be the memory used to load the segments in page cache as that doesn't count towards anonymous memory usage. So, that means the process really allocated loads of direct memory using those buffers you mentioned.
There could be a genuine memory leak there. You can identify that by doing a manual GC (e.g. using "jcmd  GC.run" )  at time of high direct memory use, if GC frees up a whole bunch of direct memory then there is almost certainly a memory leak and we are not "closing/cleaning" discarded off-heap buffers. If you find that, you can try to then identify which buffers are we not cleaning up and/or create another ticket with your task log and analysis , someone else from community might be able to look into it.
If GC does not free significant direct memory that means all that memory is really in use and code needs optimization to reduce its memory usage. My suspicion is that, most of that off-heap buffers are probably being used to merge the intermediate persisted segment merging , playing with some of configuration described in https://druid.apache.org/docs/latest/ingestion/native_tasks.html#tuningconfig might increase/decrease off heap memory usage.  (especially maxRowsInMemory, maxBytesInMemory, maxTotalRows )
Check whether segment handoffs are working fine, you can add "pushTimeout" setting from above if need be.
you might be hitting issue described in https://github.com/apache/incubator-druid/pull/6699 , if you can build druid with that patch applied then try that.
That said, please attache the task log of the process that ends up dead due to too much memory usage. I will take a quick look to see if there is something obvious that shows up.
HTH.

@himanshug Thank you so much for the quick replay!
I had performed a manual GC with using jcmd GC.run, but did not free up the direct memory.
The task log of the process that ends up dead due to too much memory usage as below.
Thanks !
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6865}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6866}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6867}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6868}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6869}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6870}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6871}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6872}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6873}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6874}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6875}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6876}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6877}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6878}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6879}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6880}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6881}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6882}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6883}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6884}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6885}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6886}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6887}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6888}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6889}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6890}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6891}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6892}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6893}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6894}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6895}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6896}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6897}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6898}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6899}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6900}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6901}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6902}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6903}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6904}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6905}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6906}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6907}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6908}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6909}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6910}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6911}]
2019-06-14T06:24:10,907 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6912}]
2019-06-14T06:24:10,908 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6913}]
2019-06-14T06:24:10,908 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6914}]
2019-06-14T06:24:10,908 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6915}]
2019-06-14T06:24:10,908 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6916}]
2019-06-14T06:24:10,908 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6917}]
2019-06-14T06:24:10,908 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6918}]
2019-06-14T06:24:10,908 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6919}]
2019-06-14T06:24:10,908 INFO [appenderator_merge_0] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Adding hydrant[FireHydrant{, queryable=ZJK_TEST_2019-06-13T06:00:00.000Z_2019-06-13T12:00:00.000Z_2019-06-14T06:17:24.368Z, count=6920}]
2019-06-14T06:24:11,090 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMergerV9 - Using SegmentWriteOutMediumFactory[TmpFileSegmentWriteOutMediumFactory]
2019-06-14T06:24:11,091 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMergerV9 - Completed version.bin in 0 millis.
2019-06-14T06:24:11,091 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMergerV9 - Completed factory.json in 0 millis
2019-06-14T06:24:11,415 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMerger - Allocating dictionary merging direct buffer with size[20]
2019-06-14T06:24:11,416 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMerger - Allocating dictionary merging direct buffer with size[12]
2019-06-14T06:24:11,419 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMerger - Allocating dictionary merging direct buffer with size[4]
2019-06-14T06:24:11,419 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMerger - Allocating dictionary merging direct buffer with size[4]
2019-06-14T06:24:11,419 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMerger - Allocating dictionary merging direct buffer with size[4]
2019-06-14T06:24:13,376 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMerger - Allocating dictionary merging direct buffer with size[168]
2019-06-14T06:24:13,376 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMerger - Allocating dictionary merging direct buffer with size[168]
2019-06-14T06:24:13,376 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMerger - Allocating dictionary merging direct buffer with size[64]
****************** many many many Allocating dictionary merging direct buffer with size [*] *******************
2019-06-14T06:24:13,987 INFO [appenderator_merge_0] org.apache.druid.segment.StringDimensionMergerV9 - Completed dim[uri] conversions with cardinality[1,084] in 722 millis.
2019-06-14T06:24:13,988 INFO [appenderator_merge_0] org.apache.druid.segment.IndexMergerV9 - Completed dim conversions in 2,596 millis.
2019-06-14T06:24:20,682 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[53]
2019-06-14T06:24:20,682 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[54]
2019-06-14T06:24:20,683 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[55]
2019-06-14T06:24:20,683 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[56]
2019-06-14T06:24:20,683 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[57]
2019-06-14T06:24:20,683 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[58]
2019-06-14T06:24:20,683 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[59]
2019-06-14T06:24:20,683 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[60]
****************** many many many Allocating new littleEndByteBuf [*] until 333,029 *******************
2019-06-14T06:24:34,446 INFO [appenderator_merge_0] org.apache.druid.segment.CompressedPools - Allocating new littleEndByteBuf[333,029]

so there were 6920 segments to be merged together with 80 columns each, merging that does need alot of those buffers, about 6920*80 . I think there are too many intermediate persists because of maxBytesInMemory which in your case , by default, is 2g/6 = 341M . OnHeapIncrementalIndex does overestimate the size due to sketches in there and might be persisting segments too aggressively.
Here is what I would try out...
set maxBytesInMemory = Integer.MAX_VALUE (to effectively disable it)
adjust maxRowsInMemory (default 1000000) to a lower value or increase Xmx on task process so that it doesn't get into too much of GC . You might have to adjust druid.worker.capacity=10 to a lower value if Xmx was increased. Now find the right balance of maxRowsInMemory and Xmx on task process so that you don't end up with thousands of segments to merge in the end but much lower number. That will drive down the off-heap buffer allocation .
(Higher maxRowsInMemory means lower the number of segments to merge in the end but higher jvm heap utilization )

@himanshug Thank you so much!
I had set maxBytesInMemory = Integer.MAX_VALUE in my task and the number of segments reduce to 1500.
Finally, the task process works very well. direct memory used 4GB(originally used almost 20GB) when the process handoff segments.
Thank you!
Have a good day!

likely related #6743

https://github.com/apache/incubator-druid/pull/7919 should further reduce the memory utilization . however that PR is independent and I am closing this bug.


Add Tracker formats

Adds searching support for XM, ImpulseTracker, and twenty other formats supported by ModPlug (and therefore LÖVE).

LÖVE Wiki
ModPlug README

I guess I don't see much point, but if LÖVE supports it, I don't see why not.


Not clear diagrams are clickable

I never realised the diagrams were slides that could be clicked through. I wonder if this could be made clearer somehow? It could well be my fault for skim-reading though.

Hmm, actually there are dots at the bottom, so it is mainly me. I wonder if the diagrams should automatically progress through the slides?

 I flicked through at speed and thought I'd missed a bunch of images, but I think most aren't animations.


Binding custom person form has errors and won't let me create or edit anymore after update

production.ERROR: Argument 1 passed to LaravelEnso\People\Http\Controllers\Create::__invoke() must be an instance of LaravelEnso\People\Forms\Builders\PersonForm, instance of App\Forms\Builders\PersonForm given {"userId":2,"exception":"[object] (TypeError(code: 0): Argument 1 passed to LaravelEnso\People\Http\Controllers\Create::__invoke() must be an instance of LaravelEnso\People\Forms\Builders\PersonForm, instance of App\Forms\Builders\PersonForm given at /home/genealogia/public_html/vendor/laravel-enso/people/src/Http/Controllers/Create.php:10)
I am binding and it worked before. The only difference is I'm trying to manually add multi tenancy code to the table builder and forms as overwriting the route didn't work for me, unless I did that wrong?

You probably forgot you extend the person-form from the package.

Can you explain what I did wrong? PersonForm doesn't extend it has just been binded as per docs? Does it need to extend too?

Changed to class PersonForm extends \LaravelEnso\People\Forms\Builders\PersonForm and still has error.

Oh, now it says about being compatiable.

production.ERROR: Declaration of App\Forms\Builders\PersonForm::edit(App\Person $person) should be compatible with LaravelEnso\People\Forms\Builders\PersonForm::edit(LaravelEnso\People\Models\Person $person) {"userId":2,"exception":"[object] (ErrorException(code: 0): Declaration of App\Forms\Builders\PersonForm::edit(App\Person $person) should be compatible with LaravelEnso\People\Forms\Builders\PersonForm::edit(LaravelEnso\People\Models\Person $person) at /home/genealogia/public_html/app/Forms/Builders/PersonForm.php:29)

My App\Person doesn't extend the Person model as it errored about cascade morph map so I had to copy all the functions instead and extend Model. Is this why?

I'm now extending default Person model after removing duplicate MorphCascade trait and same error on form about compatible $person

This is my Person:
`<?php
namespace App;
use LaravelEnso\Companies\Models\Company;
use App\Models\User;
use App\Family;
use App\Place;
use App\PersonEvent;
use LaravelEnso\DynamicMethods\Traits\Relations;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\SoftDeletes;
use Illuminate\Notifications\RoutesNotifications;
use Illuminate\Support\Collection;
use LaravelEnso\Addresses\Traits\Addressable;
use LaravelEnso\Helpers\Traits\AvoidsDeletionConflicts;
use LaravelEnso\Multitenancy\Traits\SystemConnection;
use LaravelEnso\People\Enums\Genders;
use LaravelEnso\People\Enums\Titles;
use LaravelEnso\Rememberable\Traits\Rememberable;
use LaravelEnso\Tables\Traits\TableCache;
use LaravelEnso\TrackWho\Traits\CreatedBy;
use LaravelEnso\TrackWho\Traits\UpdatedBy;
class Person extends \LaravelEnso\People\Models\Person
{
use Addressable,
AvoidsDeletionConflicts,
CreatedBy,
Relations,
Rememberable,
RoutesNotifications,
TableCache,
UpdatedBy,
SoftDeletes,
SystemConnection;
     public function __construct(Array $attributes = [])
    {
        parent::__construct($attributes);
    }

/**
 * The attributes that should be mutated to dates.
 *
 * @var array
 */
protected $dates = ['deleted_at', 'birthday','deathday'];

protected $guarded = ['id'];

protected $fillable = [
    'gid',
    'givn',
    'surn',
    'sex',
    'child_in_family_id',
    'description',
    'title', 'name', 'appellative', 'uid', 'email', 'phone', 'birthday',
    'deathday', 'bank', 'bank_account',
    'uid', 'chan', 'rin', 'resn', 'rfn', 'afn',
];

public function events()
{
    return $this->hasMany(PersonEvent::class);
}

public function child_in_family()
{
    return $this->belongsTo(Family::class, 'child_in_family_id');
}

public function husband_in_family()
{
    return $this->hasMany(Family::class, 'husband_id');
}

public function wife_in_family()
{
    return $this->hasMany(Family::class, 'wife_id');
}

public function fullname()
{
    return $this->givn.' '.$this->surn;
}

public function getSex()
{
    if ($this->sex == 'F') {
        return 'Female';
    }

    return 'Male';
}

public static function getList()
{
    $persons = self::get();
    $result = [];
    foreach ($persons as $person) {
        $result[$person->id] = $person->fullname();
    }

    return collect($result);
}

public function addEvent($title, $date, $place, $description = '')
{
    $place_id = Place::getIdByTitle($place);
    $event = PersonEvent::updateOrCreate(
        [
            'person_id' => $this->id,
            'title' => $title,
        ],
        [
            'person_id' => $this->id,
            'title' => $title,
            'description' => $description,
        ]);

    if ($date) {
        $event->date = $date;
        $event->save();
    }

    if ($place) {
        $event->places_id = $place_id;
        $event->save();
    }

    // add birthyear to person table ( for form builder )
    if ($title == 'BIRT' && ! empty($date)) {
        $this->birthday = date('Y-m-d', strtotime($date));
    }
    // add deathyear to person table ( for form builder )
    if ($title == 'DEAT' && ! empty($date)) {
        $this->deathday = date('Y-m-d', strtotime($date));
    }
    $this->save();

    return $event;
}

public function birth()
{
    return $this->events->where('title', '=', 'BIRT')->first();
}

public function death()
{
    return $this->events->where('title', '=', 'DEAT')->first();
}

public function appellative()
{
    return $this->givn;
}

protected $touches = ['user'];

public function user()
{
    return $this->hasOne(\LaravelEnso\Core\Models\User::class);
}

public function companies()
{
    return $this->belongsToMany(Company::class)
        ->withPivot(['position', 'is_main']);
}

public function hasUser()
{
    return $this->user()->exists();
}

public function company()
{
    return $this->companies()->wherePivot('is_main', true)->first();
}

public function gender()
{
    if (! $this->title) {
        return;
    }

    return $this->title === Titles::Mr
        ? Genders::Male
        : Genders::Female;
}

public function position(Company $company)
{
    return $this->companies()
        ->wherePivot('company_id', $company->id)
        ->first()->pivot->position;
}

public function syncCompanies($companyIds, $mainCompanyId)
{
    $pivotIds = (new Collection($companyIds))
        ->reduce(fn ($pivot, $value) => $pivot->put($value, [
            'is_main' => $value === $mainCompanyId,
            'is_mandatary' => false,
        ]), new Collection());

    $this->companies()->sync($pivotIds->toArray());
}

}
`

@curtisdelicata has been this solved?


Spring Petclinic application do not run....

Hi,
In an attempt to run the "Spring Petclinic" application on Web, the url simply does not work. I have tried all the options - here is the gist :

Have build the image and container for Spring Petclinic application build through a Dockerfile using Maven installed in the VM
Have pushed the image onto Docker Hub for public accessibility in my Dockerhub account
Apache2 and Tomcat 8.5.x is up and running with firewall allowed for Apache Full (80) and Tomcat (8080)
to run on browser -> http:///spring-petclinic
Above url does not work.
Please find the attached snapshots - if you can find where I am going wrong..

Here are the snapshots for ease of reference :
Dockerfile-Spring-Petclinic

apache-tomcat running

spring-petclinic container running

spring-petclinic image pushed to Dockerhub

apache tomcat url running

Network Security group config in Azure for VM

spring-petclinic url not working

run like this
http://<ip:8080>
ensure your 8080 port  is open or not.

I have taken your hint and deployed the .jar file in the correct Docker layer (this was the point of confusion, but now understand the layer structure)
and post that have used ENTRYPOINT to execute the .jar file
Here's the modified Dockerfile:
FROM maven:3-jdk-8 as mvn
LABEL author='Debasis'
RUN git clone https://github.com/spring-projects/spring-petclinic.git
RUN cd spring-petclinic  &&  mvn install -DskipTests
FROM tomcat:8
LABEL author='Debasis'
COPY --from=mvn  /spring-petclinic/target/spring-petclinic*.jar /usr/local/tomcat/webapps/spring-petclinic.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/usr/local/tomcat/webapps/spring-petclinic.jar"]

The above build is working fine now - have checked the image and the container with docker inspect.
I could get to invoke the petclinic application by
http://public ip-addr:8080/
Thanks for your response
Debasis

Hi,
I have commented in the GitHub.
The problem is resolved
Thanks & Regards,
Debasis Pal7972930164 /<PHONE_NUMBER>
On Sunday, 8 March 2020, 12:23:36 GMT+5:30, srmanikanta<EMAIL_ADDRESS>wrote:

run like this
http://ip:8080
ensure your 8080 port is open or not.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.


Issue 79 add urllib3 to requirements file

Added the urllib3 1.25.11 to fix the incompatibility with the requests library

Codecov Report

Merging #80 (c707255) into master (6d63d3c) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master       #80   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            2         2
  Lines          200       200
=========================================
  Hits           200       200

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d63d3c...c707255. Read the comment docs.

IBM Cloud toolchain: Delivery Pipeline deployed products to dev, including commit 41a997cd7bc4c7e16f2e861045947fcbbb7d90f2

IBM Cloud toolchain: Delivery Pipeline deployed products to prod, including commit 41a997cd7bc4c7e16f2e861045947fcbbb7d90f2


Fix #2433: Configure automatic CLA check as a GitHub Action.

Inspired by https://github.com/scalameta/scalameta/pull/1906

This probably needs to be merged to master before it takes effect. So we can probably not test this before it reaches master ...


[Feature] (Willing to PR) Avoid KV cache occupying GPU memory when not used

Checklist

[X] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
[X] 2. Please use English, otherwise it will be closed.

Motivation
Hi thank you for the library! The use case is that, when doing online PPO, I hope to use SGLang to generate llm completions, and then use RL to do gradient descent on those completions.
The problem is, to do this on a single GPU, the timeline is "SGLang generate - Torch backward - repeat it". Thus, when torch doing backprop, I hope SGLang can free its KV cache memory consumption, otherwise torch will not have enough memory.
Thanks for any suggestions!
Related resources
No response

Thanks for pointing this out. I am working on similar things on OpenRLHF.
https://github.com/OpenRLHF/OpenRLHF
Off-loading KV Cache (actually you should shut down the engine and runtime indeed) has a clear trade-off:

Pros:

It saves a lot of VRAM.
Cons:

Relaunching the engine takes a lot of time. This time comes from two parts:

Saving the updated weights to disk.
Relaunching the engine.

In the current design of OpenRLHF, they choose not to offload to save time. In this case, the weights can be directly broadcast from the training engine to the inference engine. As shown in:
api: https://github.com/sgl-project/sglang/blob/21e9e63ad56f8bd25663fa6907ed92f47a2b2724/python/sglang/srt/server.py#L214-L239
test case / usage: https://github.com/sgl-project/sglang/blob/main/test/srt/test_update_weights_from_distributed.py
If you feel interested, welcome to continue discussing this with me on github or on our SGLang slack.

@zhaochenyang20 Hi thank you so much! My current thought is that, maybe we do not shutdown the engine at all, thus we do not need to slowly restart engine. Instead, some naive thoughts:

We somehow avoid the kv cache from occupying GPU memory. For example, can we del the_kv_cache_torch_tensors and re-create them later? This can be done on the cache part, and all other parts in sglang do not need to know about this. Because when there are zero requests to sglang engine, all other parts should be sleeping and do not touch the kv cache, thus they will never know the cache tensors are removed and later re-created.
2a. As for model weights, is it even possible that we have only exactly one copy of model memory, shared by both SGLang and Transformers model (used by TRL / OpenRLHF to do weight updates)?
2b. Or, for model weights, can we do the same thing as the proposal in "1.", i.e. temporarily delete the tensors but remain all other parts, and later re-create the tensors from OpenRLHF's new weighhts?

In my experience, I do not think 2 is possible. But maybe 1 is okay. Let me discuss this with my teammates. @fzyzcjy

@zhaochenyang20 Thank you! I am happy to PR and try to hack it as well.

Btw I see https://github.com/OpenRLHF/OpenRLHF/pull/614 and it looks great :)

@zhaochenyang20 Quickly glanced at the code, it seems calling https://github.com/sgl-project/sglang/blob/23e5e50fd5fba7f315e04294f55060a8171fcc69/python/sglang/srt/mem_cache/memory_pool.py#L212 when we want to free memory (and call _create_buffers again when we want to use sglang later) may be related.

Quick experiments about devices:
model = transformers.AutoModelForCausalLM.from_pretrained('...')
print(model.device, model) # cpu

model.to('cuda')
print(model.device, model) # gpu, and see GPU memory become high

model.to('meta')
torch.cuda.empty_cache()
print(model.device, model) # meta, and see GPU memory usage become low again

model.to_empty(device='cuda')
print(model.device, model) # gpu, and GPU memory high again

Thus for point "2b", maybe we do not need to really delete the tensors, but only need to do a llama_model.to('meta') and later llama_model.to_empty(device='cuda') + llama_model.update_weights().
For point "1", doing this instead of deleing the whole tensor may also be an alternative way.

@fzyzcjy Hey there. I connect you through WeChat. We can have a quick discussion.

I made a quick hack and it seems to work.
Code
Note: I hack the function get_weights_by_name to allow me to call between processes for this quick experiment. In a real PR, surely we should create some new functions. Also, in real PR, we should make _clear_buffers public, etc.
Change scheduler.py
    def get_weights_by_name(self, recv_req: GetWeightsByNameReqInput):
        print(f'hi hacky change get_weights_by_name!!! {recv_req=}')
        match recv_req.name:
            case 'hack_pause':
                self.flush_cache()
                self.token_to_kv_pool._clear_buffers()
                torch.cuda.empty_cache()
            case 'hack_resume':
                self.token_to_kv_pool._create_buffers()
            case _:
                raise NotImplementedError

        return None

Test code
import sglang as sgl

llm = sgl.Engine(
    model_path="meta-llama/Llama-3.2-1B-Instruct",
    # model_path='Qwen/Qwen2.5-0.5B-Instruct',
    # enable_torch_compile=False,  # temp try
    disable_cuda_graph=True,  # temp for debug
)
print(llm)

prompts = [
    "1+1=",
]

sampling_params = {"temperature": 0}

print('llm.generate #1')
outputs = llm.generate(prompts, sampling_params)
print(outputs)

print('pause')
llm.get_weights_by_name('hack_pause')

print('sleep for seconds...')
time.sleep(3)

print('resume')
llm.get_weights_by_name('hack_resume')

print('llm.generate #2')
outputs = llm.generate(prompts, sampling_params)
print(outputs)

Results:
The red color is GPU memory occupied. As we can see, during the 3 second sleep, the GPU memory is much lower, since we seem to successfully free the kv cache pool.

Another proposal: https://github.com/sgl-project/sglang/issues/2569

@fzyzcjy Hey. Amazing experiments! Funny that get_weights_by_name is implemented by me, to check the correctness of the weight update.
api: https://github.com/sgl-project/sglang/blob/21e9e63ad56f8bd25663fa6907ed92f47a2b2724/python/sglang/srt/server.py#L214-L239
test case / usage: https://github.com/sgl-project/sglang/blob/main/test/srt/test_update_weights_from_distributed.py

Also, regarding your proposed method, in real implementation, we should definitely have some functions like realse_gpu_occpation and resume_gpu_occupation, like what you hacked in get_weights_by_name. I think the current stage is pretty cool. If you'd like to PR, I am pretty willing to review!!!
@fzyzcjy

https://github.com/sgl-project/sglang/issues/2569
@fzyzcjy I also relied on this. And, do you have time for a Zoom/Tencent meeting recently? We can have a discussion at your convenience. I assume you are in China, so the morning and noon in China is appropriate for both of us.

Sure! We should have separate function names like that instead of hacking an unrelated function. I am happy to PR :)

@fzyzcjy We will PR this into OpenRLHF. Do you think trl is also needed? We (SGLang and lmsys.org) are willing to collaborate with HuggingFace also 😂

I guess it is up to your (SGLang and lmsys)'s choice. My personal thoughts is that, it seems many people are using TRL, so it may be great to PR to that.
Btw, https://github.com/sgl-project/sglang/issues/2569 is not only restricted to be useful to OpenRLHF and TRL, but also for Transformers, since the proposal is about a speedup to PreTrainedModel.generate using SGLang.

I guess it is up to your (SGLang and lmsys)'s choice. My personal thoughts is that, it seems many people are using TRL, so it may be great to PR to that.
Btw, #2569 is not only restricted to be useful to OpenRLHF and TRL, but also for Transformers, since the proposal is about a speedup to PreTrainedModel.generate using SGLang.

Pretty good! Let's discuss this later. @fzyzcjy

The major concern of this approach is that cuda graph will be disabled. torch.compile is only useful when using cuda graph - this is the current design in sglang.
When sampling with vllm:
[5] NVIDIA H100 80GB HBM3 | 29°C,   0 % | 37858 / 81559 MB | chenyang(21218M) chenyang(16530M)
[6] NVIDIA H100 80GB HBM3 | 32°C,   0 % | 47336 / 81559 MB | chenyang(31702M) chenyang(15528M)
[7] NVIDIA H100 80GB HBM3 | 29°C,   0 % | 74861 / 81559 MB | chenyang(74844M)

When doing weights update:
[5] NVIDIA H100 80GB HBM3 | 30°C,   0 % | 57074 / 81559 MB | chenyang(40434M) chenyang(16530M)
[6] NVIDIA H100 80GB HBM3 | 33°C,   0 % | 45832 / 81559 MB | chenyang(30196M) chenyang(15530M)
[7] NVIDIA H100 80GB HBM3 | 29°C,   0 % | 74861 / 81559 MB | chenyang(74844M)

The major concern of this approach is that cuda graph will be disabled.
torch.compile is only useful when using cuda graph - this is the current design in sglang

Ah I see. Some naive thoughts:

How much slower if we just naively disable the CUDA graph?

We save memory, but lost speed, thus maybe we are still in a Pareto frontier. For example, suppose in the 8xH100 example in #2583, we somehow enable to use only 4xH100, then as long as we are not 2x slower, we are still saving the overall money.

Is it possible we do torch.compile without using CUDA graph? For example, when using a HF model to do SFT training, it seems I always do torch.compile and never do CUDA graph capture but it works well.

(The comment above is updated adding the "brainstorm" point just now)

Oh man I am saying almost the same thing as @merrymercy in https://github.com/sgl-project/sglang/pull/2588#pullrequestreview-2523267154!

More brainstorms:

One way to implement memory allocator is to allocate a large continuous chunk of memory, and then get chunks from it. Thus we may need to hack torch's current Caching Allocator or hack RMM's allocator.
Another potential way may be to use cuMemCreate/cuMemMap/... More concretely, if the only requirement of CUDA graph is that "tensor GPU virtual memory address must be same" (have not done experiments to verify), then when we want to temporarily release the memory, maybe we can cuMemUnmap + cuMemRelease, then the virtual address is remained, while the physical memory is de-allocated. Now we have physical memory to do other allocations.

P.S. Extra potential pros: for an unmapped block, hopefully (not tested yet) CUDA can throw error on it, then users/we get for free some error messages if we get something wrong

Just quick and naive brainstorms, I have not done experiments yet.

Find a little bit of time to quickly hacked an experiment for the last brainstorm.
Results
Memory consumption is like this (x is time, red is consumed memory)

Code
example.py

import ctypes
import time
from typing import Callable

import torch

# print('change_current_allocator')
# new_alloc = torch.cuda.memory.CUDAPluggableAllocator(
#     str(Path(__file__).parent / 'my_alloc.so'), 'my_malloc', 'my_free')
# torch.cuda.memory.change_current_allocator(new_alloc)

# TODO ok?
cdll_my_preload_so = ctypes.CDLL('./my_preload.so')
print(f'{cdll_my_preload_so=}')

def _ptr(x):
    assert isinstance(x, torch.Tensor)
    return hex(x.data_ptr())

class KVCache:
    def __init__(self):
        self.create_buffers(1)

    def create_buffers(self, value):
        # or model weights, etc
        self.kv_buffer = torch.tensor([value, value, value], dtype=torch.int32, device='cuda')
        print(f'create_buffers {_ptr(self.kv_buffer)=}')

    def clear_buffers(self):
        del self.kv_buffer

    def execute(self, arg: torch.Tensor) -> torch.Tensor:
        # print(f'KVCache.execute {arg=} {self.kv_buffer=}')
        return arg + self.kv_buffer

# https://pytorch.org/blog/accelerating-pytorch-with-cuda-graphs/
def create_cuda_graph(fn: Callable):
    # warmup
    s = torch.cuda.Stream()
    s.wait_stream(torch.cuda.current_stream())
    with torch.cuda.stream(s):
        print('with torch.cuda.stream(s) execute fn')
        fn()
    torch.cuda.current_stream().wait_stream(s)

    # capture
    g = torch.cuda.CUDAGraph()
    with torch.cuda.graph(g):
        print('with torch.cuda.graph(g) execute fn')
        fn()

    return g

def run():
    cache = KVCache()
    static_input = torch.zeros((3,), dtype=torch.int32, device='cuda')
    static_output = torch.zeros((3,), dtype=torch.int32, device='cuda')
    print(f'{_ptr(static_input)=} {_ptr(static_output)=}')

    def fn():
        nonlocal static_output
        static_output = cache.execute(static_input)

    g = create_cuda_graph(fn)

    print('replay #1')
    static_input[...] = 100
    g.replay()
    print(f'{static_output=}')
    assert torch.all(
        static_output == torch.tensor([101, 101, 101], dtype=torch.int32, device='cuda')), f'{static_output=}'

    # cache.clear_buffers()

    big_tensor = torch.zeros((2_000_000_000,), dtype=torch.uint8, device='cuda')
    print(f'{big_tensor=}')

    print('torch.cuda.empty_cache()')
    torch.cuda.empty_cache()

    print('sleep...')
    time.sleep(3)

    print('call hack_release_occupation')
    cdll_my_preload_so.hack_release_occupation()

    print('sleep...')
    time.sleep(3)

    # this should fail
    # print(f'{cache.kv_buffer=}')

    print('call hack_resume_occupation')
    cdll_my_preload_so.hack_resume_occupation()

    dummy = torch.zeros((3,), device='cuda')
    print(f'{_ptr(dummy)=}')

    # cache.create_buffers(2)

    cache.kv_buffer[...] = 2

    print('replay #2')
    static_input[...] = 200
    g.replay()
    print(f'{static_output=}')
    assert torch.all(
        static_output == torch.tensor([202, 202, 202], dtype=torch.int32, device='cuda')), f'{static_output=}'

    print('sleep...')
    time.sleep(3)

    print(f'{big_tensor=}')
    print(f'{dummy=}')

if __name__ == '__main__':
    run()

my_preload.cc

#include <sys/types.h>
#include <cuda_runtime_api.h>
#include <cuda.h>
#include <iostream>
#include <stdio.h>
#include <dlfcn.h>
#include <map>

// https://stackoverflow.com/questions/6083337/overriding-malloc-using-the-ld-preload-mechanism
static cudaError_t (*real_cudaMalloc)(void**, size_t) = NULL;
static cudaError_t (*real_cudaFree)(void*) = NULL;

struct MyInfo {
    CUmemGenericAllocationHandle allocHandle;
    size_t size;
};

int currentDev = 0; // HACK

std::map<void*, MyInfo> info_of_ptr_map;

static void my_init(void) {
    real_cudaMalloc = (cudaError_t (*)(void**, size_t)) dlsym(RTLD_NEXT, "cudaMalloc");
    if (NULL == real_cudaMalloc) {
        fprintf(stderr, "Error in `dlsym`: %s\n", dlerror());
    }

    real_cudaFree = (cudaError_t (*)(void*)) dlsym(RTLD_NEXT, "cudaFree");
    if (NULL == real_cudaFree) {
        fprintf(stderr, "Error in `dlsym`: %s\n", dlerror());
    }
}

void mem_create(CUmemGenericAllocationHandle *allocHandle, size_t size) {
//    size_t granularity = 0;
    CUmemAllocationProp prop = {};
    prop.type = CU_MEM_ALLOCATION_TYPE_PINNED;
    prop.location.type = CU_MEM_LOCATION_TYPE_DEVICE;
    prop.location.id = currentDev;
//    cuMemGetAllocationGranularity(&granularity, &prop, CU_MEM_ALLOC_GRANULARITY_MINIMUM);
//    padded_size = ROUND_UP(size, granularity);
    cuMemCreate(allocHandle, size, &prop, 0);
}

void mem_set_access(void* devPtr, size_t size) {
    CUmemAccessDesc accessDesc = {};
    accessDesc.location.type = CU_MEM_LOCATION_TYPE_DEVICE;
    accessDesc.location.id = currentDev;
    accessDesc.flags = CU_MEM_ACCESS_FLAGS_PROT_READWRITE;
    cuMemSetAccess((CUdeviceptr)devPtr, size, &accessDesc, 1);
}

cudaError_t cudaMalloc(void** devPtr, size_t size) {
    if (real_cudaMalloc == NULL) {
        my_init();
    }

//    cudaError_t ret = real_cudaMalloc(devPtr, size);
//    std::cout << "[my_preload.cc] cudaMalloc" << " devPtr=" << devPtr << " size=" << size << " ret=" << ret << std::endl;
//    return ret;

    CUmemGenericAllocationHandle allocHandle;
    mem_create(&allocHandle, size);

    /* Reserve a virtual address range */
    cuMemAddressReserve((CUdeviceptr*)devPtr, size, 0, 0, 0);
    /* Map the virtual address range
     * to the physical allocation */
    cuMemMap((CUdeviceptr)*devPtr, size, 0, allocHandle, 0);

    mem_set_access(*devPtr, size);

    info_of_ptr_map[*devPtr] = MyInfo { allocHandle, size };

    std::cout << "[my_preload.cc] cudaMalloc"
        << " devPtr=" << devPtr << " size=" << size
        << " allocHandle=" << allocHandle
        << std::endl;

    return cudaSuccess;
}

cudaError_t cudaFree(void* devPtr) {
    if (real_cudaFree == NULL) {
        my_init();
    }

//    cudaError_t ret = real_cudaFree(devPtr);
//    std::cout << "[my_preload.cc] cudaFree" << " devPtr=" << devPtr << " ret=" << ret << std::endl;
//    return ret;

    MyInfo info = info_of_ptr_map[devPtr];
    info_of_ptr_map.erase(devPtr);

    cuMemUnmap((CUdeviceptr)devPtr, info.size);
    cuMemRelease(info.allocHandle);
    cuMemAddressFree((CUdeviceptr)devPtr, info.size);

    std::cout << "[my_preload.cc] cudaFree"
        << " devPtr=" << devPtr << " info.size=" << info.size
        << " info.allocHandle=" << info.allocHandle
        << std::endl;

    return cudaSuccess;
}

extern "C" {
    void hack_release_occupation() {
        for (auto it = info_of_ptr_map.begin(); it != info_of_ptr_map.end(); ++it) {
            void* devPtr = it->first;
            MyInfo info = it->second;
            cuMemUnmap((CUdeviceptr)devPtr, info.size);
            cuMemRelease(info.allocHandle);

            std::cout << "[my_preload.cc] hack_release_occupation"
                << " devPtr=" << devPtr << " info.size=" << info.size << " info.allocHandle=" << info.allocHandle
                << std::endl;
        }
    }

    void hack_resume_occupation() {
        for (auto it = info_of_ptr_map.begin(); it != info_of_ptr_map.end(); ++it) {
            void* devPtr = it->first;
            MyInfo &info = it->second;

            CUmemGenericAllocationHandle newAllocHandle;
            mem_create(&newAllocHandle, info.size);

            cuMemMap((CUdeviceptr)devPtr, info.size, 0, newAllocHandle, 0);

            mem_set_access(devPtr, info.size);

            std::cout << "[my_preload.cc] hack_resume_occupation"
                << " devPtr=" << devPtr << " info.size=" << info.size << " (old)info.allocHandle=" << info.allocHandle
                << " (new)newAllocHandle=" << newAllocHandle
                << std::endl;

            info.allocHandle = newAllocHandle;
        }
    }
}

build command
g++ my_preload.cc -o my_preload.so -shared -fPIC -lcuda -I/usr/local/cuda/include

run command
LD_PRELOAD=./my_preload.so python3 example.py

@fzyzcjy Sorry for replying late as I have some things to do these days. If I do not reply in one day, please ping me again in WeChat. Have a good day

@zhaochenyang20 No worries! I used a bit more time to dig deeper into the approach above and make it work well for SGLang: KV Cache can be released AND CUDA graph can still be enabled.
Also, theoretically speaking, this approach should make the "release model weight" very easy to implement (only two lines of code to add). But I will need to implement "update model weight from same card" feature to test that.

Find another bit of time to submit the PR: https://github.com/sgl-project/sglang/pull/2630.

By adding this single line, the memory occupation when SGLang in paused mode reduces from 4.5GB to 3.4GB.

(Debugging logits for model weights now)

Update: Model weight seems to successfully released. Now, when in paused mode, SGLang only takes 1.1GB memory for llama1B model.

Summary: When in paused mode,

KV cache is released: 22GB -> 4.5GB
ReqToTokenPool.req_token is released: 4.5GB -> 3.4GB
Model weight is released: 3.4GB -> 1.1GB

(numbers above are on my 4090D card, but the idea is similar; I will do more experiments later)

Btw, some scenarios come to my mind that may be especially powerful when one uses SGLang: When the generation phase in RL is especially large. For example, do a lot of "rollout"s, or MCTS searches.

This is pretty cool!

@merrymercy Yes, that has already been done!

Quick update: On a standard 3xH100 setting (1xH100 for actor+ref, 1xH100 for critic+reward, 1xH100 for sglang), the memory of SGLang can be released when paused and becomes 2.5GB (instead of 72.2GB). Will check the 2xH100 (do not let SGLang take extra GPU) as well as comparison tests will be handled later.

Quick update: On a standard 3xH100 setting (1xH100 for actor+ref, 1xH100 for critic+reward, 1xH100 for sglang), the memory of SGLang can be released when paused and becomes 2.5GB (instead of 72.2GB). Will check the 2xH100 (do not let SGLang take extra GPU) as well as comparison tests later.

Metrics for very short runs are as follows: (Again, correctness has not been verified)

Great job. Ping me on slack is needed!


Why might I get a timeout creating a BOSHDeployment?

$ kaf docs/examples/bosh-deployment/boshdeployment-with-custom-variable.yaml -n scf
secret/nats-deployment.var-custom-password configured
configmap/nats-manifest unchanged
configmap/ops-scale unchanged
Error from server (Timeout): error when creating "vendor/cf-operator/docs/examples/bosh-deployment/boshdeployment-with-custom-variable.yaml": Timeout: request did not complete within requested timeout 30s

During this time, there are no logs in cf-operator related to the attempted BOSHDeployment creation.
Ideas why this might happen?

I've upgraded to latest cf-operator and this issue no longer appears. I'll leave ticket open briefly in case anyone can share some reasons why this might have happened.

Getting the same thing here, even with what I believe is the latest version of the operator: v0.4.1-60.g642832d0 taken from https://github.com/starkandwayne/bootstrap-kubernetes-demos/blob/ba8960a26b63d6fa5315cbd5b23462cf34352281/bin/bootstrap-system-cf-operator#L6
The thing is: Even a complete purge of all namespaces (the operator one included) doesn't seem to help.

@bkrannich can you please try the latest version (0.4.2), and also maybe increase the contextTimeout setting, as described here: https://github.com/cloudfoundry-incubator/cf-operator/blob/master/deploy/helm/cf-operator/README.md#configuration

@viovanov Sorry, missed to reply here. Turns out, I unintentionally dropped --set "operator.watchNamespace=scf" from my helm install, so the operator wasn't listening to the namespace into which I uploaded SCF.
Unless @drnic has additional topics, I believe we can close this one.


[Obstacles] Recover after some period of time

Noticing that many units can lay obstacles everywhere, if so, we'll have no space to battle.

Solved.


What the meaning of ITER_SIZE and always get stuck during training?

Hi, When I train voc dataset using your code, it always stuck at some time (when I use 4xM40 to train my model, it always stuck in iter 282/20000), no error no crash, just got stuck, do you have any idea about this error? (miou is about 74 after train about 7 hours, which is reasonable).
and I saw you set the ITER_SIZE in voc12.yaml, what the purpose of this parameter and does it have any  connection with my error?
Thanks in advance!

For iter_size, please look at README and train.py. It is equivalent to iter_size in Caffe.

BVLC/caffe wiki
Accumulate gradients across batches through the iter_size solver field. With this setting batch_size: 16 with iter_size: 1 and batch_size: 4 with iter_size: 4 are equivalent.

The purpose is to save a video memory, although you might be able to set BATCH_SIZE: 10, ITER_SIZE: 1 with your GPUs. I'm not sure about your stucking issue but I have ever seen such reports with the multi-GPU environment which is due to NCCL problem.

On my machine, the code has succeeded to train with two Titan X. Just for your information.

@kazuto1011
Thanks for your reply! I solved this bug, it caused by opencv in dataloader, you can check the details in this issue: pytorch/pytorch#1355, just add cv2.setNumThreads(0) in  init function of the dataloader.
Thanks for your codes and kind help again! :)


ng lint warnings cannot read property of undefined

Bug Report or Feature Request (mark with an x)
- [x] bug report
- [ ] feature request

Versions.
@angular/cli: 1.0.0-rc.4
node: 7.7.3
os: linux x64
@angular/common: 2.4.10
@angular/compiler: 2.4.10
@angular/core: 2.4.10
@angular/forms: 2.4.10
@angular/http: 2.4.10
@angular/platform-browser: 2.4.10
@angular/platform-browser-dynamic: 2.4.10
@angular/router: 3.4.10
@angular/cli: 1.0.0-rc.4
@angular/compiler-cli: 2.4.10
Repro steps.
ng new test123
cd test123
ng lint
-> all ok
add "no-unnecessary-qualifier": true and "promise-function-async": true to tslint.json
ng lint --type-check
The log given by the failure.
Warning: Cannot read property 'exports' of undefined
Warning: Cannot read property 'flags' of undefined
Warning: Cannot read property 'members' of undefined

Desired functionality.
No warnings.
Mention any other details that might be useful.
When I use webpack 2.3.1, tslint 4.5.1 and codelyzer 2.0.1 without angular-cli I get no warnings with this rules.

Sounds like a bug alright, thanks for the report!

I get this warning when using tslint-loader in a Webpack project, but there's no Angular involved. Seems harmless. At least I haven't noticed what effect it has, if any.

ng lint seems to work, just ng lint --type-check shows those errors. Also when I run tslint directly (even with --type-check I don't get those warnings

I see
Warning: Cannot read property 'some' of undefined
when enabling no-unnecessary-qualifier rule. I'm running tslint directly. tslint 5.1.0, codelyzer 3.0.1. Maybe it is related to this one.

Tried to replicate on new project seems to be working fine. with below configuration.
@angular/cli: 1.1.0-beta.0
node: 7.5.0

devDependencies
    "@angular/cli": "1.1.0-beta.0",
    "@angular/compiler-cli": "^4.0.0",
    "@angular/language-service": "^4.0.0",
    "@types/jasmine": "2.5.38",
    "@types/node": "~6.0.60",
    "codelyzer": "~3.0.0",
    "jasmine-core": "~2.5.2",
    "jasmine-spec-reporter": "~3.2.0",
    "karma": "~1.4.1",
    "karma-chrome-launcher": "~2.1.1",
    "karma-cli": "~1.0.1",
    "karma-jasmine": "~1.1.0",
    "karma-jasmine-html-reporter": "^0.2.2",
    "karma-coverage-istanbul-reporter": "^1.0.0",
    "protractor": "~5.1.0",
    "ts-node": "~2.0.0",
    "tslint": "~5.1.0",
    "typescript": "~2.3.1"

I'm also confirming that I don't see the (maybe-related) issue with no-unnecessary-qualifier. I don't remember exactly if it was fixed by upgrading tslint to 5.2, but it was some component upgrade that fixed this warning.

Yes it seems bo be fixed.


Update "Who's Organizing This?"

idk if we should list out EVERYONE's names now? I could add another sentence "x y z have joined our team to help run 2019" since they're also not wics folks (and then keep first sentence the same) but idk if that sounds bad

I think listing everyone's name is a nice thing to do (see !!con's site).

That sounds good to me! I can do when I get back or let Chirag or someone else do beforehand if you think it's more urgent


!jobs - List jobs in CareerHub

Similar to !events.
Probably scrape off "https://www.careerhub.uq.edu.au/students/jobs/search?query=&typeofwork=-1&location=&country=Any&occupation=916&residency=&page=1&take=" + amount
For more than 10 PM them.
Possibly add caching

Could we do internship application (for tech companies) dates too?

Good idea, but will need a way to keep track of them, maybe a google calendar?
Day suggested implementing the Seek API aswell which seems like a good idea


limit presale quantity?

thanks for the stuffs. i'm new to this stuff and i hope i can do this by myself but i'm not. can you please guide me how to automatically pauses minting after reaching specific amount of supply like ...to.be.revertedWith('Max preSale supply exceeded!');? to avoid wholesale then not reaching public sale for a small number of collection. sorry for my bad english hope you get the point, also sorry if this was unnecessary question. thanks in advance!

Hi @DedeFawzy this thing can be done, but it would require some work on both the contract and the DAPP in order to have a good UX (my main concern is: let's say you can mint 3 tokens per transaction during presale and the presale limit is 1000 while the current supply is 999, the DAPP would allow people to try minting up to 3 tokens, but it would fail because they can mint only 1...).
Anyway I'm sorry but I don't have time to work on this kind of custom example right now... I'm gonna leave this open so maybe someone else can help (or I can get back on this if I have some time).
Thank you for your understanding!

Hi @fawzydede, since many users started using issues for help/feature requests instead of bug reports, I decided to open a dedicated Discussions section. This will help me focusing on security issues and bugs while the whole community can grow and develop a strong know-how together.
Discussions will be divided in categories with different purposes and features that should fit any need, I encourage you moving this topic there if you are still interested in it. 😉
Thank you!


feat: polls as a cast type

What is the feature you would like to implement?
Allow polls where a user can post a question with multiple options and other users can make a choice.
Why is this feature important?
A common user behavior that will be important in any public social network. It is particularly important to get right early because the data format is likely to be very different from a regular cast.
Will the protocol spec need to be updated??
May need a new replication type in Section 4
How should this feature be built? (optional)
TBD
Additional context
TBD

@varunsrin are you sure this should be part of v2.0.0? Other than the profile metadata CRDT, I didn't think we wanted to add new message types for v2.0.0.

I don't think we should ship this in v2.0.0 but i think its worth designing now to understand how it might fit into a CRDT and then punting the implementation to post v2.0


CAMEL-17962: fix build error caused by cleanup of duplicated charset features in camel-netty

Merging it directly, as it is causing a build failure on some of the Netty code.


Change token file path to  $HOME/.serverless

It has a risk that sending tokens to a remote repositories.
ref #1

Released as v0.0.5.


Polymorphic effect aliases

I'd like to write this:
effect Iterate[A] {def emit(element: A): Unit}
effect Reduce[A] {def consume(): Option[A]}
effect Transduce[A, B] = {Reduce[A], Iterate[B]}

But I get a parse error pointing at the = in Transduce[A, B] = {...} saying '{' expected but '=' found. Is this intentional? It looks more like a parsing oversight than an actual limitation.

In deed, when working on the new effect tracking mechanism for closures, I temporarily dropped polymorphic effect aliases. They "just" need to be added back, there is no theoretical reason for not having them.
Sorry if this is inconvenient at the moment.


fixes problem of small f64 values being treated as integers

This will now cast a value such as 2e-06 to a float, rather than an integer despite the lack of a decimal point. Values such as 2e06 are still treated as integer.

Good catch - I will release a patch version with this.


Moved VMware vSphere driver to main b2d release tree

vSphere driver can be moved to the b2d main branch as it only depends on VMware tools that have been merged already.

LGTM


Add Netflix Vizercal

A way to visualize traffic inside the Mesh

+1

Yah! That would be awesome!

I am going to close this issue as now we have Kiali


Add Ænix to adopters

@gecube could you please rebase, thanks!

@stefanprodan done, thanks for the attention

Ænix was already added to the adopters list earlier this year, about 8 entries back!

@kingdonb my bad .... I have a memory like guppy fish :-(

It's ok, I am excited about Cozystack too, easy mistake to make!


Broken hyperlinks

Hello, I found multiple broken hyperlinks. Some of them are relative links that don't work:

[ ] grammarly (file: README.md, status code: 404)
[ ] crazy-max (file: README.md, status code: 404)
[ ] deepfence (file: README.md, status code: 404)
[ ] istio (file: README.md, status code: 404)
[ ] g31s (file: README.md, status code: 404)
[ ] https://github.com/prologic/autodock (file: README.md, status code: 404)
[ ] Sandbox (file: README.md, status code: 404)
[ ] ahmetalpbalkan (file: README.md, status code: 404)
[ ] https://github.com/konstruktoid/Docker/blob/master/Security/CheatSheet.adoc (file: README.md, status code: 404)
[ ] http://docker-br.herokuapp.com (file: README.md, status code: 404)

Thanks for the hint @ndsvw. We will improve it.


Move middleware stack exectution from router to middleware-dispatcher …

Q
A

Is bugfix?
❌

New feature?
❌

Breaks BC?
❌

Merge after https://github.com/yiisoft/middleware-dispatcher/pull/2

👍