Get Metrics Function

POST 
/get_metrics_function

Get the function to generate the metrics given results (partially supported)

Request

application/json

Body

required

dataset Dataset (string)required

config

object

required

custom_extract_logic: a piece of python code that calls submit_code_blocks(cbs) to extract custom code cbs: List[CodeBlock], CodeBlock(priority=40, code='xxx', language='xxx') priority: fenced = 30, incomplete fenced = 20, heuristic = 10

dataset_type

object

the dataset class used to process, only works when the dataset id is not registered.

anyOf

MOD1

string

language

object

anyOf

MOD1

string

Possible values: [python, cpp, nodejs, go, go_test, java, php, csharp, bash, typescript, sql, rust, cuda, lua, R, perl, D_ut, ruby, scala, julia, pytest, junit, kotlin_script, jest, verilog, python_gpu, lean, swift, racket]

locale

object

anyOf

MOD1

string

is_fewshot

object

anyOf

MOD1

boolean

compile_timeout

object

anyOf

MOD1

number

run_timeout

object

anyOf

MOD1

number

custom_extract_logic

object

anyOf

MOD1

string

provided_data

object

anyOf

MOD1

Array [

property name*

object

anyOf

MOD1

string

MOD2

integer

]

MOD2

property name*

object

anyOf

MOD1

string

MOD2

integer

extra object

Default value: {}

Responses

200

Successful Response

application/json

Schema

Schema

function

object

required

anyOf

MOD1

string

Example (from schema)

{
  "function": "string"
}

422

Validation Error

application/json

Schema

Schema

detail

object[]

Array [

loc

object[]

required

Array [

anyOf

MOD1

string

MOD2

integer

]

msg Message (string)required

type Error Type (string)required

]

Example (from schema)

{
  "detail": [
    {
      "loc": [
        "string",
        0
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}

Loading...