Sub-group Communication

7. Sub-group Communication#

7.1. fred#

Perform a floating-point reduction operation across participating work-items within a sub-group.

Syntax:

fred.op<.nanp>.type dst, src0, src1

  .op = { .min, .max, .absmax }
  .type = { .bf, .hf, .f }


  - dst, src0 are registers
  - src1 can be a 32-bit register or an immediate

Restrictions

  • bitwidth of dst and src0 must match the bitwidth of .type

PISA Notes

  • introduced in PISA version 0.1

  • requires PISA target 100 or compatible

packed:

fred.op<.nanp>.type dst, src0, src1

  .op = { .min, .max, .absmax }
  .type = { .bfx2, .hfx2 }


  - dst, src0 are 2x16-bit vector registers
  - src1 can be a 32-bit register or an immediate

PISA Notes

  • introduced in PISA version 0.1

  • requires PISA target 100 or compatible

Semantics:

allParticipatingLanes = src1 & %activemask
if ((1 << %laneid) & allParticipatingLanes == 0)
  return

reduce(.op, src0, allParticipatingLanes)

reduce(Op, Src, ParticipatingLanes) {
  switch Op {
    case .min:
      dst = fmin<.nanp>(Src of work-items in ParticipatingLanes)
      break
    case .max:
      dst = fmax<.nanp>(Src of work-items in ParticipatingLanes)
      break
    case .absmax:
      dst = fmax<.nanp>(fabs(Src of work-items in ParticipatingLanes))
      break
  }
}

Notes:

src1 is a sub-group-uniform bit mask that selects the participating work-items, with each bit position corresponding to a work-item’s %laneid. Non-participating work-items do not contribute to the reduction operation, and do not update their dst values.

By default, NaN inputs are suppressed during the reduction: a NaN contributed by a participating work-item is ignored unless every participating work-item contributes a NaN, in which case a quiet NaN is produced.

When .nanp is specified, NaN inputs propagate. If any participating work-item contributes a NaN of any type, the reduction result is a quiet NaN.

Examples:

.reg .32b %dst, %src0;

// maximum of single-float %src0 across all active work-items
fred.max.f   %dst, %src0, 0xffffffff;
// minimum of single-float %src0 across active work-items with laneid 16-31
fred.min.f   %dst, %src0, 0xffff0000;
// maximum of single-float with NaN propagation
fred.max.nanp.f   %dst, %src0, 0xffffffff;
// element-wise maximum of packed bfloat vector across active work-items with laneid 0-15
.reg .v2.16b %dst_packed, %src0_packed;
fred.max.bfx2  %dst_packed, %src0_packed, 0xffff;

7.2. ired#

Perform an integer reduction operation across participating work-items within a sub-group.

Syntax:

ired.op.type dst, src0, src1

  .op = { .sum, .smin, .smax, .umin, .umax, .and, .or, .xor, .absmax }
  .type = { .16b, .32b }


  - dst, src0 are registers
  - src1 can be a 32-bit register or an immediate

Restrictions

  • bitwidth of dst and src0 must match the bitwidth of .type

PISA Notes

  • introduced in PISA version 0.1

  • requires PISA target 100 or compatible

Semantics:

allParticipatingLanes = src1 & %activemask
if ((1 << %laneid) & allParticipatingLanes == 0)
  return

reduce(.op, src0, allParticipatingLanes)

reduce(Op, Src, ParticipatingLanes) {
  switch Op {
    case .sum:
      dst = sum(Src of work-items in ParticipatingLanes)
      break
    case .smin:
      dst = smin(Src of work-items in ParticipatingLanes)
      break
    case .smax:
      dst = smax(Src of work-items in ParticipatingLanes)
      break
    case .umin:
      dst = umin(Src of work-items in ParticipatingLanes)
      break
    case .umax:
      dst = umax(Src of work-items in ParticipatingLanes)
      break
    case .and:
      dst = and(Src of work-items in ParticipatingLanes)
      break
    case .or:
      dst = or(Src of work-items in ParticipatingLanes)
      break
    case .xor:
      dst = xor(Src of work-items in ParticipatingLanes)
      break
    case .absmax:
      dst = umax(iabs(Src of work-items in ParticipatingLanes))
      break
  }
}

Notes:

src1 is a sub-group-uniform bit mask that selects the participating work-items, with each bit position corresponding to a work-item’s %laneid. Non-participating work-items do not contribute to the reduction operation, and do not update their dst values.

Important

For the .absmax operation, the most negative value representable in the specified .type is interpreted as the largest positive value; for example, when .type == .32b, 0x80000000 is treated as the maximum value.

Examples:

.reg .32b %dst, %src0;

// sum of %src0 across all active work-items
ired.sum.32b   %dst, %src0, 0xffffffff;
// signed max of %src0 across active work-items with laneid 0-15
ired.smax.32b  %dst, %src0, 0xffff;

7.3. redfirstidx#

Get the index of the first active work-item within a sub-group.

Syntax:

redfirstidx.type dst, src0

  .type = { .32b }


  - dst is a 32-bit register
  - src0 can be a 32-bit register or an immediate

PISA Notes

  • introduced in PISA version 0.1

  • requires PISA target 100 or compatible

Semantics:

participatingLanes = src0 & %activemask

dst = 0
while ((participatingLanes & 1) == 0) {
  dst++
  participatingLanes >>= 1
}

Notes:

src0 is a sub-group-uniform bit mask that selects the participating work-items, with each bit position corresponding to a work-item’s %laneid.

Examples:

.reg .32b %dst;

// index of the first active work-item across active work-items with laneid 0-15
redfirstidx.32b   %dst, 0xffff;

7.4. shfl#

Shuffle data across work-items within a sub-group.

Syntax:

shfl.op<.sg>.type dst, src0, src1, src2

  .op = { .up, .dn, .xor, .idx }
  .type = { .32b }


  - dst, src0 are 32-bit registers
  - src1, src2 can be 32-bit registers or immediates

PISA Notes

  • introduced in PISA version 0.1

  • requires PISA target 100 or compatible

Semantics:

participatingLanes = src2 & %activemask
if ((1 << %laneid) & participatingLanes == 0)
  return

laneOffset = src1[4:0]
if (.sg) {
  cval = src1[12:8]
  segmask = src1[20:16]
} else {
  cval = (.op == .up) ? 0x0 : 0x1F
  segmask = 0x0
}

clampLane = (%laneid & segmask) | (cval & ~segmask)
minLane = (%laneid & segmask)

switch .op{
  case .up:
    srcLane = %laneid - laneOffset
    break
  case .dn:
    srcLane = %laneid + laneOffset
    break
  case .xor:
    srcLane = %laneid ^ laneOffset
    break
  case .idx:
    srcLane = minLane | (laneOffset & ~segmask)
    break
}
inBounds = (.op == .up) ? (srcLane >= clampLane) : (srcLane <= clampLane)
if (!inBounds || !participatingLanes[srcLane]) {
  dst = src0[%laneid]
} else {
  dst = src0[srcLane]
}

Notes:

src1 specifies lane offset in bits [4:0], clamp value in bits [12:8], and segmask in bits [20:16]. Each c bit in the table below can be 0 or 1, and is used to encode the clamp value.

Segmask and Clamp Value Encoding#

segmask

sub-group width

cval

11110

2

1111c

11100

4

111cc

11000

8

11ccc

10000

16

1cccc

00000

32

ccccc

The qualifier .sg is an optional qualifier that indicates that this is a sub-group shuffle operation. If not specified, the instruction defaults to a full-group shuffle operation - equivalent to having 1 sub-group inclusive of all work-items - and bits [20:8] of src1 are ignored.

src2 is a sub-group-uniform bit mask that selects the participating work-items, with each bit position corresponding to a work-item’s %laneid. Non-participating work-items do not contribute to the shuffle operation, and do not update their dst values.

Important

Out-of-bounds work-item access copies its own src0 to dst.

Examples:

.reg .32b %dst, %src0, %src1;

// shuffle data across all active work-items with idx op
shfl.idx.32b   %dst, %src0, %src1, 0xffffffff;

// shuffle data up across active work-items with laneid 16-31
shfl.up.32b   %dst, %src0, 1, 0xffff;


.reg .32b %value, %sum, %tmp;

// prefix sum across sub-group of size 32:
// sum: N-1
shfl.up.32b   %tmp, %value, 1, 0xffffffff;
// sum: sum(N, N-1)
iadd.32b   %sum, %value, %tmp;
// tmp: N-2, N-3
shfl.up.32b   %tmp, %sum, 2, 0xffffffff;
// sum: sum(N, N-1, N-2, N-3)
iadd.32b   %sum, %sum, %tmp;
// tmp: sum(N-4, N-5, N-6, N-7)
shfl.up.32b   %tmp, %sum, 4, 0xffffffff;
// sum: sum(N, N-1, N-2, ..., N-7)
iadd.32b   %sum, %sum, %tmp;
// tmp: sum(N-8, N-9, ..., N-15)
shfl.up.32b   %tmp, %sum, 8, 0xffffffff;
// sum: sum(N, N-1, N-2, ..., N-15)
iadd.32b   %sum, %sum, %tmp;
// tmp: sum(N-16, N-17, ..., N-31)
shfl.up.32b   %tmp, %sum, 16, 0xffffffff;
// sum: sum(N, N-1, N-2, ..., N-31)
iadd.32b   %sum, %sum, %tmp;
// sum: sum(N-1, N-2, ..., N-31)
isub.32b   %sum, %sum, %value;

.reg .32b %flag, %expr, %r1, %r2, %r3, %r4, %r5, %result;
.pred %p;

// check if %expr is the same for every work-item in the sub-group
// Shuffle from immediate lower neighbor
shfl.up.32b   %r1, %expr, 1, 0xffffffff;
ucmp.ne.32b   %p, %r1, %expr;
sel.32b  %flag, 0x0, 0x1, %p;
// Now perform reduction
shfl.xor.32b   %r1, %flag, 16, 0xffffffff;
iadd.32b   %r1, %r1, %flag;
shfl.xor.32b   %r2, %r1, 8, 0xffffffff;
iadd.32b   %r2, %r2, %r1;
shfl.xor.32b  %r3, %r2, 4, 0xffffffff;
iadd.32b   %r3, %r3, %r2;
shfl.xor.32b  %r4, %r3, 2, 0xffffffff;
iadd.32b   %r4, %r4, %r3;
shfl.xor.32b  %r5, %r4, 1, 0xffffffff;
iadd.32b   %result, %r5, %r4;

ucmp.eq.32b   %p, %result, 0;
goto.cond   %p, all_equal;
goto.cond   !%p, not_all_equal;
not_all_equal:
all_equal:

// Sub-group shuffle data down by 2 with sub-thread width of 8
// (laneoffset = 5b00010, cval = 5b00000, segmask=5b11000)
// %a |31|30|29|28|27|26|25|24|23|22|21|20|19|18|17|16|15|..
// %r |  |  |31|30|29|28|27|26|  |  |23|22|21|20|19|18|  |..

.reg .32b %r, %a;
shfl.dn.sg.32b %r, %a, 0x00180002, 0xffffffff;

// Sub-group shuffle data down by 2 with sub-thread width of 8
// (laneoffset = 5b00010, cval = 5b11101, segmask=5b11000)
// %a |31|30|29|28|27|26|25|24|23|22|21|20|19|18|17|16|15|..
// %r |  |  |  |  |29|28|27|26|  |  |  |  |21|20|19|18|  |..

.reg .32b %r, %a;
shfl.dn.sg.32b %r, %a, 0x00181d02, 0xffffffff;
Contents